interest together with data from a language pair
that has a typologically-similar source language
and more training data (i.e. Russian and Belarusian, Turkish and Azerbaijani). The results under
“Ours” are our many-to-one and many-to-many
models we trained identically in terms of model
architecture and hyper-parameters.
We ﬁrst note that our many-to-many model outperforms all other models when translating into
English, with 1.82 BLEU improvement (when averaged across the four language pairs) over the
best ﬁne-tuned many-to-one models of Neubig
and Hu (2018) and 2.44 BLEU improvement over
our many-to-one model when averaged across the
four low-resource language pairs (Table 1). This
is surprising as it uses the same X !En data,
model architecture and capacity as our many-toone model, while handling a heavier burden since
it also supports 58 additional translation tasks
(from English into58 languages). Our models also
outperform the more complex models of Wang
et al. (2019) which use ”Soft Decoupled Encoding” for the input tokens, while our models use a
simple subword segmentation.
Multilingual neural machine translation (NMT) enables training a single model
that supports translation from multiple source languages into multiple target
languages. In this paper, we push the limits of multilingual NMT in terms of
number of languages being used. We perform extensive experiments in training
massively multilingual NMT models, translating up to 102 languages to and from
English within a single model. We explore different setups for training such
models and analyze the trade-offs between translation quality and various
modeling decisions. We report results on the publicly available TED talks
multilingual corpus where we show that massively multilingual many-to-many
models are effective in low resource settings, outperforming the previous
state-of-the-art while supporting up to 59 languages. Our experiments on a
large-scale dataset with 102 languages to and from English and up to one
million examples per direction also show promising results, surpassing strong
bilingual baselines and encouraging future work on massively multilingual NMT.
Roee Aharoni Melvin Johnson Orhan Firat