Neural speech-to-text translation

In this project, we are interested to translate speech to text. This can be used for translating a speaker who speaks in French to English in order to make it understandable by an English speaking person.

There are various aspects to this project, including (but not limited to): (i) handling multi-modal speech and text data, and (ii) producing simultaneous translations. This project involves innovating interesting models to address different aspects of this problem (eg [1,2]).

[1] Sequence to Sequence Mixture Model for Diverse Machine Translation
Xuanli He, Gholamreza Haffari, Mohammad Norouzi
In Proceedings of CoNLL, 2018.

[2] Automatic Post-Editing of Machine Translation: A Neural Programmer-Interpreter Approach
Trang Vu, Gholamreza Haffari
In Proceedings of EMNLP, 2018.