As with the previous example, the RNNencdec began deviating from the actual meaning of the
source sentence after generating approximately 30 words (see the underlined phrase). After that
point, the quality of the translation deteriorates, with basic mistakes such as the lack of a closing
quotation mark.
Again, the RNNsearch-50 was able to translate this long sentence correctly:
Ce genre d’exp ´erience fait partie des efforts de Disney pour ”prolonger la dur ´ee
de vie de ses s ´eries et cr ´eer de nouvelles relations avec des publics viades
plateformes num´eriques deplus enplus importantes”, a-t-il ajout ´e.
In conjunction with the quantitative results presented already, these qualitative observations conﬁrm our hypotheses that the RNNsearch architecture enables far more reliable translation of long
sentences than the standard RNNencdec model.
In Appendix C, we provide a few more sample translations of long source sentences generated by
the RNNencdec-50, RNNsearch-50 and Google Translate along with the reference translations.
6 R ELATED WORK
6.1 L EARNING TO ALIGN
A similar approach of aligning an output symbol with an input symbol was proposed recently by
Neural machine translation is a recently proposed approach to machine
translation. Unlike the traditional statistical machine translation, the neural
machine translation aims at building a single neural network that can be
jointly tuned to maximize the translation performance. The models proposed
recently for neural machine translation often belong to a family of
encoder-decoders and consists of an encoder that encodes a source sentence into
a fixed-length vector from which a decoder generates a translation. In this
paper, we conjecture that the use of a fixed-length vector is a bottleneck in
improving the performance of this basic encoder-decoder architecture, and
propose to extend this by allowing a model to automatically (soft-)search for
parts of a source sentence that are relevant to predicting a target word,
without having to form these parts as a hard segment explicitly. With this new
approach, we achieve a translation performance comparable to the existing
state-of-the-art phrase-based system on the task of English-to-French
translation. Furthermore, qualitative analysis reveals that the
(soft-)alignments found by the model agree well with our intuition.
Dzmitry Bahdanau Kyunghyun Cho Yoshua Bengio