4.2 Multilinguality & Zero-Shot
Performance
We then study the effect of the number of languages on zero-shot translation accuracy. Since
we ﬁnd zero-shot accuracy as an interesting measure for model generalization, we hypothesize that
by adding more languages, the model is forced to
create a more generalized representation to better utilize its capacity, which may improve zeroshot performance. We choose four language pairs
for this purpose: Arabic $French which are distant languages, and Ukrainian $Russian which are
similar. Table 8 shows the results of our models
on these language pairs. For Arabic $French the
BLEU scores are very low in all cases, with the
50-to-50 and 25-to-25 models being slightly better than rest on Ar-Fr and Fr-Ar respectively. On
Russian $Ukrainian we see clear improvements
when increasing the number of languages to more
than ﬁve.
Figure 2 further illustrates this, showing the better generalization performance of the massively
multilingual models under this zero-shot setting.
While the zero-shot performance in this case is
low and unstable for the 5-to-5 and 25-to-25 modupdate 5-to-5 25-to-25 50-to-50 75-to-75 103-to-103
0 0 0 0 0 0
Multilingual neural machine translation (NMT) enables training a single model
that supports translation from multiple source languages into multiple target
languages. In this paper, we push the limits of multilingual NMT in terms of
number of languages being used. We perform extensive experiments in training
massively multilingual NMT models, translating up to 102 languages to and from
English within a single model. We explore different setups for training such
models and analyze the trade-offs between translation quality and various
modeling decisions. We report results on the publicly available TED talks
multilingual corpus where we show that massively multilingual many-to-many
models are effective in low resource settings, outperforming the previous
state-of-the-art while supporting up to 59 languages. Our experiments on a
large-scale dataset with 102 languages to and from English and up to one
million examples per direction also show promising results, surpassing strong
bilingual baselines and encouraging future work on massively multilingual NMT.
Roee Aharoni Melvin Johnson Orhan Firat