Barnes-hut-sne. In Proceedings of the First International Conference on Learning Representations
(ICLR 2013) , May.
[Vaswani et al.2013] Ashish Vaswani, Yinggong Zhao,
Victoria Fossum, and David Chiang. 2013. Decoding with large-scale neural language models improves translation. Proceedings of the Conference
on Empirical Methods in Natural Language Processing , pages 1387–1392.
[Zeiler2012] Matthew D. Zeiler. 2012. ADADELTA:
an adaptive learning rate method. Technical report,
arXiv 1212.5701.
[Zou et al.2013] Will Y . Zou, Richard Socher,
Daniel M. Cer, and Christopher D. Manning.
2013. Bilingual word embeddings for phrase-based
machine translation. In Proceedings of the ACL
Conference on Empirical Methods in Natural
Language Processing (EMNLP) , pages 1393–1398.
A RNN Encoder–Decoder
In this document, we describe in detail the architecture of the RNN Encoder–Decoder used in the experiments.
Let us denote an source phrase by X= (x1;x2;:::;xN)and a target phrase by Y=
In this paper, we propose a novel neural network model called RNN
Encoder-Decoder that consists of two recurrent neural networks (RNN). One RNN
encodes a sequence of symbols into a fixed-length vector representation, and
the other decodes the representation into another sequence of symbols. The
encoder and decoder of the proposed model are jointly trained to maximize the
conditional probability of a target sequence given a source sequence. The
performance of a statistical machine translation system is empirically found to
improve by using the conditional probabilities of phrase pairs computed by the
RNN Encoder-Decoder as an additional feature in the existing log-linear model.
Qualitatively, we show that the proposed model learns a semantically and
syntactically meaningful representation of linguistic phrases.
Kyunghyun Cho Bart van Merrienboer Caglar Gulcehre Dzmitry Bahdanau Fethi Bougares Holger Schwenk Yoshua Bengio