space language models. Comput. Speech Lang. ,
21(3):492–518, July.
[Schwenk2012] Holger Schwenk. 2012. Continuous
space translation models for phrase-based statistical machine translation. In Martin Kay and Christian Boitet, editors, Proceedings of the 24th International Conference on Computational Linguistics
(COLIN) , pages 1071–1080.
[Socher et al.2011] Richard Socher, Eric H. Huang, Jeffrey Pennington, Andrew Y . Ng, and Christopher D.
Manning. 2011. Dynamic pooling and unfolding
recursive autoencoders for paraphrase detection. In
Advances in Neural Information Processing Systems
24.
[Son et al.2012] Le Hai Son, Alexandre Allauzen, and
Franc ¸ois Yvon. 2012. Continuous space translation models with neural networks. In Proceedings of
the 2012 Conference of the North American Chapter of the Association for Computational Linguistics:
Human Language Technologies , NAACL HLT ’12,
pages 39–48, Stroudsburg, PA, USA.
[van der Maaten2013] Laurens van der Maaten. 2013.
Barnes-hut-sne. In Proceedings of the First International Conference on Learning Representations
(ICLR 2013) , May.
In this paper, we propose a novel neural network model called RNN
Encoder-Decoder that consists of two recurrent neural networks (RNN). One RNN
encodes a sequence of symbols into a fixed-length vector representation, and
the other decodes the representation into another sequence of symbols. The
encoder and decoder of the proposed model are jointly trained to maximize the
conditional probability of a target sequence given a source sequence. The
performance of a statistical machine translation system is empirically found to
improve by using the conditional probabilities of phrase pairs computed by the
RNN Encoder-Decoder as an additional feature in the existing log-linear model.
Qualitatively, we show that the proposed model learns a semantically and
syntactically meaningful representation of linguistic phrases.
Kyunghyun Cho Bart van Merrienboer Caglar Gulcehre Dzmitry Bahdanau Fethi Bougares Holger Schwenk Yoshua Bengio