et al. (2014) used this approach to generate translations from their neural machine translation model.
For more details on the architectures of the models and training procedure used in the experiments,
see Appendices A and B.
5 R ESULTS
5.1 Q UANTITATIVE RESULTS
In Table 1, we list the translation performances measured in BLEU score. It is clear from the table
that in all the cases, the proposed RNNsearch outperforms the conventional RNNencdec. More
importantly, the performance of the RNNsearch is as high as that of the conventional phrase-based
translation system (Moses), when only the sentences consisting of known words are considered.
This is a signiﬁcant achievement, considering that Moses uses a separate monolingual corpus (418M
words) in addition to the parallel corpora we used to train the RNNsearch and RNNencdec.
6We used the tokenization script from the open-source machine translation package, Moses.
7In this paper, by a ’hidden unit’, we always mean the gated hidden unit (see Appendix A.1.1).
5
Published as a conference paper at ICLR 2015
The
agreement
on
the
European
Economic
Area
was
signed
in
August
1992
.
<end>
L'
accord
Neural machine translation is a recently proposed approach to machine
translation. Unlike the traditional statistical machine translation, the neural
machine translation aims at building a single neural network that can be
jointly tuned to maximize the translation performance. The models proposed
recently for neural machine translation often belong to a family of
encoder-decoders and consists of an encoder that encodes a source sentence into
a fixed-length vector from which a decoder generates a translation. In this
paper, we conjecture that the use of a fixed-length vector is a bottleneck in
improving the performance of this basic encoder-decoder architecture, and
propose to extend this by allowing a model to automatically (soft-)search for
parts of a source sentence that are relevant to predicting a target word,
without having to form these parts as a hard segment explicitly. With this new
approach, we achieve a translation performance comparable to the existing
state-of-the-art phrase-based system on the task of English-to-French
translation. Furthermore, qualitative analysis reveals that the
(soft-)alignments found by the model agree well with our intuition.
Dzmitry Bahdanau Kyunghyun Cho Yoshua Bengio