J. Mach. Learn. Res. ,3, 1137–1155.
Bergstra, J., Breuleux, O., Bastien, F., Lamblin, P., Pascanu, R., Desjardins, G., Turian, J., WardeFarley, D., and Bengio, Y . (2010). Theano: a CPU and GPU math expression compiler. In
Proceedings of the Python for Scientiﬁc Computing Conference (SciPy) . Oral Presentation.
Boulanger-Lewandowski, N., Bengio, Y ., and Vincent, P. (2013). Audio chord recognition with
recurrent neural networks. In ISMIR .
Cho, K., van Merrienboer, B., Gulcehre, C., Bougares, F., Schwenk, H., and Bengio, Y . (2014a).
Learning phrase representations using RNN encoder-decoder for statistical machine translation.
InProceedings of the Empiricial Methods in Natural Language Processing (EMNLP 2014) . to
appear.
Cho, K., van Merri ¨enboer, B., Bahdanau, D., and Bengio, Y . (2014b). On the properties of neural
machine translation: Encoder–Decoder approaches. In Eighth Workshop on Syntax, Semantics
Neural machine translation is a recently proposed approach to machine
translation. Unlike the traditional statistical machine translation, the neural
machine translation aims at building a single neural network that can be
jointly tuned to maximize the translation performance. The models proposed
recently for neural machine translation often belong to a family of
encoder-decoders and consists of an encoder that encodes a source sentence into
a fixed-length vector from which a decoder generates a translation. In this
paper, we conjecture that the use of a fixed-length vector is a bottleneck in
improving the performance of this basic encoder-decoder architecture, and
propose to extend this by allowing a model to automatically (soft-)search for
parts of a source sentence that are relevant to predicting a target word,
without having to form these parts as a hard segment explicitly. With this new
approach, we achieve a translation performance comparable to the existing
state-of-the-art phrase-based system on the task of English-to-French
translation. Furthermore, qualitative analysis reveals that the
(soft-)alignments found by the model agree well with our intuition.
Dzmitry Bahdanau Kyunghyun Cho Yoshua Bengio