Quirk, and Geoffrey Zweig. 2013. Joint language
and translation modeling with recurrent neural networks. In Proceedings of the ACL Conference on
Empirical Methods in Natural Language Processing
(EMNLP) , pages 1044–1054.
[Axelrod et al.2011] Amittai Axelrod, Xiaodong He,
and Jianfeng Gao. 2011. Domain adaptation via
pseudo in-domain data selection. In Proceedings of
the ACL Conference on Empirical Methods in Natural Language Processing (EMNLP) , pages 355–362.
[Bastien et al.2012] Fr ´ed´eric Bastien, Pascal Lamblin,
Razvan Pascanu, James Bergstra, Ian J. Goodfellow,
Arnaud Bergeron, Nicolas Bouchard, and Yoshua
Bengio. 2012. Theano: new features and speed improvements. Deep Learning and Unsupervised Feature Learning NIPS 2012 Workshop.
[Bengio et al.2003] Yoshua Bengio, R ´ejean Ducharme,
Pascal Vincent, and Christian Janvin. 2003. A neural probabilistic language model. J. Mach. Learn.
Res., 3:1137–1155, March.
In this paper, we propose a novel neural network model called RNN
Encoder-Decoder that consists of two recurrent neural networks (RNN). One RNN
encodes a sequence of symbols into a fixed-length vector representation, and
the other decodes the representation into another sequence of symbols. The
encoder and decoder of the proposed model are jointly trained to maximize the
conditional probability of a target sequence given a source sequence. The
performance of a statistical machine translation system is empirically found to
improve by using the conditional probabilities of phrase pairs computed by the
RNN Encoder-Decoder as an additional feature in the existing log-linear model.
Qualitatively, we show that the proposed model learns a semantically and
syntactically meaningful representation of linguistic phrases.
Kyunghyun Cho Bart van Merrienboer Caglar Gulcehre Dzmitry Bahdanau Fethi Bougares Holger Schwenk Yoshua Bengio