ordered differently between French and English, and we see an example in Fig. 3 (a). From this
ﬁgure, we see that the model correctly translates a phrase [European Economic Area] into [zone
´economique europ ´een]. The RNNsearch was able to correctly align [zone] with [Area], jumping
over the two words ([European] and [Economic]), and then looked one word back at a time to
complete the whole phrase [zone ´economique europ ´eenne].
The strength of the soft-alignment, opposed to a hard-alignment, is evident, for instance, from
Fig. 3 (d). Consider the source phrase [the man] which was translated into [l’ homme]. Any hard
alignment will map [the] to [l’] and [man] to [homme]. This is not helpful for translation, as one
must consider the word following [the] to determine whether it should be translated into [le], [la],
[les] or [l’]. Our soft-alignment solves this issue naturally by letting the model look at both [the] and
[man], and in this example, we see that the model was able to correctly translate [the] into [l’]. We
Neural machine translation is a recently proposed approach to machine
translation. Unlike the traditional statistical machine translation, the neural
machine translation aims at building a single neural network that can be
jointly tuned to maximize the translation performance. The models proposed
recently for neural machine translation often belong to a family of
encoder-decoders and consists of an encoder that encodes a source sentence into
a fixed-length vector from which a decoder generates a translation. In this
paper, we conjecture that the use of a fixed-length vector is a bottleneck in
improving the performance of this basic encoder-decoder architecture, and
propose to extend this by allowing a model to automatically (soft-)search for
parts of a source sentence that are relevant to predicting a target word,
without having to form these parts as a hard segment explicitly. With this new
approach, we achieve a translation performance comparable to the existing
state-of-the-art phrase-based system on the task of English-to-French
translation. Furthermore, qualitative analysis reveals that the
(soft-)alignments found by the model agree well with our intuition.
Dzmitry Bahdanau Kyunghyun Cho Yoshua Bengio