the existing phrase-based statistical machine translation. It is a striking result, considering that the
proposed architecture, or the whole family of neural machine translation, has only been proposed
as recently as this year. We believe the architecture proposed here is a promising step toward better
machine translation and a better understanding of natural languages in general.
One of challenges left for the future is to better handle unknown, or rare words. This will be required
for the model to be more widely used and to match the performance of current state-of-the-art
machine translation systems in all contexts.
9
Published as a conference paper at ICLR 2015
ACKNOWLEDGMENTS
The authors would like to thank the developers of Theano (Bergstra et al. , 2010; Bastien et al. ,
2012). We acknowledge the support of the following agencies for research funding and computing
support: NSERC, Calcul Qu ´ebec, Compute Canada, the Canada Research Chairs and CIFAR. Bahdanau thanks the support from Planet Intelligent Systems GmbH. We also thank Felix Hill, Bart van
Merri ´enboer, Jean Pouget-Abadie, Coline Devin and Tae-Ho Kim.
REFERENCES
Axelrod, A., He, X., and Gao, J. (2011). Domain adaptation via pseudo in-domain data selection.
Neural machine translation is a recently proposed approach to machine
translation. Unlike the traditional statistical machine translation, the neural
machine translation aims at building a single neural network that can be
jointly tuned to maximize the translation performance. The models proposed
recently for neural machine translation often belong to a family of
encoder-decoders and consists of an encoder that encodes a source sentence into
a fixed-length vector from which a decoder generates a translation. In this
paper, we conjecture that the use of a fixed-length vector is a bottleneck in
improving the performance of this basic encoder-decoder architecture, and
propose to extend this by allowing a model to automatically (soft-)search for
parts of a source sentence that are relevant to predicting a target word,
without having to form these parts as a hard segment explicitly. With this new
approach, we achieve a translation performance comparable to the existing
state-of-the-art phrase-based system on the task of English-to-French
translation. Furthermore, qualitative analysis reveals that the
(soft-)alignments found by the model agree well with our intuition.
Dzmitry Bahdanau Kyunghyun Cho Yoshua Bengio