show respectively the scores on all the sentences and,
on the sentences without any unknown word in themselves and in the reference translations. Note that
RNNsearch-50?was trained much longer until the
performance on the development set stopped improving. () We disallowed the models to generate [UNK]
tokens when only the sentences having no unknown
words were evaluated (last column).
5.2 Q UALITATIVE ANALYSIS
5.2.1 A LIGNMENT
The proposed approach provides an intuitive way to inspect the (soft-)alignment between the words
in a generated translation and those in a source sentence. This is done by visualizing the annotation
weightsijfrom Eq. (6), as in Fig. 3. Each row of a matrix in each plot indicates the weights
associated with the annotations. From this we see which positions in the source sentence were
considered more important when generating the target word.
We can see from the alignments in Fig. 3 that the alignment of words between English and French
is largely monotonic. We see strong weights along the diagonal of each matrix. However, we also
observe a number of non-trivial, non-monotonic alignments. Adjectives and nouns are typically
ordered differently between French and English, and we see an example in Fig. 3 (a). From this
Neural machine translation is a recently proposed approach to machine
translation. Unlike the traditional statistical machine translation, the neural
machine translation aims at building a single neural network that can be
jointly tuned to maximize the translation performance. The models proposed
recently for neural machine translation often belong to a family of
encoder-decoders and consists of an encoder that encodes a source sentence into
a fixed-length vector from which a decoder generates a translation. In this
paper, we conjecture that the use of a fixed-length vector is a bottleneck in
improving the performance of this basic encoder-decoder architecture, and
propose to extend this by allowing a model to automatically (soft-)search for
parts of a source sentence that are relevant to predicting a target word,
without having to form these parts as a hard segment explicitly. With this new
approach, we achieve a translation performance comparable to the existing
state-of-the-art phrase-based system on the task of English-to-French
translation. Furthermore, qualitative analysis reveals that the
(soft-)alignments found by the model agree well with our intuition.
Dzmitry Bahdanau Kyunghyun Cho Yoshua Bengio