Furthermore, qualitative analysis reveals that the proposed model ﬁnds a linguistically plausible
(soft-)alignment between a source sentence and the corresponding target sentence.
2 B ACKGROUND : NEURAL MACHINE TRANSLATION
From a probabilistic perspective, translation is equivalent to ﬁnding a target sentence ythat maximizes the conditional probability of ygiven a source sentence x, i.e., arg maxyp(yjx). In
neural machine translation, we ﬁt a parameterized model to maximize the conditional probability
of sentence pairs using a parallel training corpus. Once the conditional distribution is learned by a
translation model, given a source sentence a corresponding translation can be generated by searching
for the sentence that maximizes the conditional probability.
Recently, a number of papers have proposed the use of neural networks to directly learn this conditional distribution (see, e.g., Kalchbrenner and Blunsom, 2013; Cho et al. , 2014a; Sutskever et al. ,
2014; Cho et al. , 2014b; Forcada and ˜Neco, 1997). This neural machine translation approach typically consists of two components, the ﬁrst of which encodes a source sentence xand the second
decodes to a target sentence y. For instance, two recurrent neural networks (RNN) were used by
Neural machine translation is a recently proposed approach to machine
translation. Unlike the traditional statistical machine translation, the neural
machine translation aims at building a single neural network that can be
jointly tuned to maximize the translation performance. The models proposed
recently for neural machine translation often belong to a family of
encoder-decoders and consists of an encoder that encodes a source sentence into
a fixed-length vector from which a decoder generates a translation. In this
paper, we conjecture that the use of a fixed-length vector is a bottleneck in
improving the performance of this basic encoder-decoder architecture, and
propose to extend this by allowing a model to automatically (soft-)search for
parts of a source sentence that are relevant to predicting a target word,
without having to form these parts as a hard segment explicitly. With this new
approach, we achieve a translation performance comparable to the existing
state-of-the-art phrase-based system on the task of English-to-French
translation. Furthermore, qualitative analysis reveals that the
(soft-)alignments found by the model agree well with our intuition.
Dzmitry Bahdanau Kyunghyun Cho Yoshua Bengio