Hebrew (He) and Italian (It).
2.2 Model Details
Using the same data, we trained three massively
multilingual models: a many-to-many model
which we train using all 116 translation directions
with 58 languages to-and-from English, a one-tomany model from English into 58 languages, and
a many-to-one model from 58 languages into English. We follow the method of Ha et al. (2016);
Johnson et al. (2017) and add a target-language
1github.com/neulab/
word-embeddings-for-nmt
2Chinese, Czech, English, Estonian, Finnish, French,
German, Hindi, Hungarian, Latvian, Romanian, Russian,
Spanish, Turkish. According to http://www.statmt.
org/wmtXX
preﬁx token to each source sentence to enable
many-to-many translation. These different setups
enable us to examine the effect of the number of
translation tasks on the translation quality as measured in BLEU (Papineni et al., 2002). We also
compare our massively multilingual models to
bilingual baselines and to two recently published
results on this dataset (Neubig and Hu (2018);
Wang et al. (2019)).
Multilingual neural machine translation (NMT) enables training a single model
that supports translation from multiple source languages into multiple target
languages. In this paper, we push the limits of multilingual NMT in terms of
number of languages being used. We perform extensive experiments in training
massively multilingual NMT models, translating up to 102 languages to and from
English within a single model. We explore different setups for training such
models and analyze the trade-offs between translation quality and various
modeling decisions. We report results on the publicly available TED talks
multilingual corpus where we show that massively multilingual many-to-many
models are effective in low resource settings, outperforming the previous
state-of-the-art while supporting up to 59 languages. Our experiments on a
large-scale dataset with 102 languages to and from English and up to one
million examples per direction also show promising results, surpassing strong
bilingual baselines and encouraging future work on massively multilingual NMT.
Roee Aharoni Melvin Johnson Orhan Firat