[man], and in this example, we see that the model was able to correctly translate [the] into [l’]. We
observe similar behaviors in all the presented cases in Fig. 3. An additional beneﬁt of the soft alignment is that it naturally deals with source and target phrases of different lengths, without requiring a
counter-intuitive way of mapping some words to or from nowhere ([NULL]) (see, e.g., Chapters 4
and 5 of Koehn, 2010).
5.2.2 L ONG SENTENCES
As clearly visible from Fig. 2 the proposed model (RNNsearch) is much better than the conventional
model (RNNencdec) at translating long sentences. This is likely due to the fact that the RNNsearch
does not require encoding a long sentence into a ﬁxed-length vector perfectly, but only accurately
encoding the parts of the input sentence that surround a particular word.
As an example, consider this source sentence from the test set:
An admitting privilege is the right of a doctor to admit a patient to a hospital or
a medical centre tocarry outadiagnosis oraprocedure, based onhisstatus asa
health care worker atahospital.
The RNNencdec-50 translated this sentence into:
Neural machine translation is a recently proposed approach to machine
translation. Unlike the traditional statistical machine translation, the neural
machine translation aims at building a single neural network that can be
jointly tuned to maximize the translation performance. The models proposed
recently for neural machine translation often belong to a family of
encoder-decoders and consists of an encoder that encodes a source sentence into
a fixed-length vector from which a decoder generates a translation. In this
paper, we conjecture that the use of a fixed-length vector is a bottleneck in
improving the performance of this basic encoder-decoder architecture, and
propose to extend this by allowing a model to automatically (soft-)search for
parts of a source sentence that are relevant to predicting a target word,
without having to form these parts as a hard segment explicitly. With this new
approach, we achieve a translation performance comparable to the existing
state-of-the-art phrase-based system on the task of English-to-French
translation. Furthermore, qualitative analysis reveals that the
(soft-)alignments found by the model agree well with our intuition.
Dzmitry Bahdanau Kyunghyun Cho Yoshua Bengio