Lewis. 2010. Intelligent selection of language
model training data. In Proceedings of the ACL
2010 Conference Short Papers , ACLShort ’10,
pages 220–224, Stroudsburg, PA, USA.[Pascanu et al.2014] R. Pascanu, C. Gulcehre, K. Cho,
and Y . Bengio. 2014. How to construct deep recurrent neural networks. In Proceedings of the Second
International Conference on Learning Representations (ICLR 2014) , April.
[Saxe et al.2014] Andrew M. Saxe, James L. McClelland, and Surya Ganguli. 2014. Exact solutions
to the nonlinear dynamics of learning in deep linear neural networks. In Proceedings of the Second
International Conference on Learning Representations (ICLR 2014) , April.
[Schwenk et al.2006] Holger Schwenk, Marta R. CostaJuss`a, and Jos ´e A. R. Fonollosa. 2006. Continuous
space language models for the iwslt 2006 task. In
IWSLT , pages 166–173.
[Schwenk2007] Holger Schwenk. 2007. Continuous
space language models. Comput. Speech Lang. ,
21(3):492–518, July.
[Schwenk2012] Holger Schwenk. 2012. Continuous
In this paper, we propose a novel neural network model called RNN
Encoder-Decoder that consists of two recurrent neural networks (RNN). One RNN
encodes a sequence of symbols into a fixed-length vector representation, and
the other decodes the representation into another sequence of symbols. The
encoder and decoder of the proposed model are jointly trained to maximize the
conditional probability of a target sequence given a source sequence. The
performance of a statistical machine translation system is empirically found to
improve by using the conditional probabilities of phrase pairs computed by the
RNN Encoder-Decoder as an additional feature in the existing log-linear model.
Qualitatively, we show that the proposed model learns a semantically and
syntactically meaningful representation of linguistic phrases.
Kyunghyun Cho Bart van Merrienboer Caglar Gulcehre Dzmitry Bahdanau Fethi Bougares Holger Schwenk Yoshua Bengio