  This work studies the long-standing problems of model capacity and negative
interference in multilingual neural machine translation MNMT. We use network
pruning techniques and observe that pruning 50-70% of the parameters from a
trained MNMT model results only in a 0.29-1.98 drop in the BLEU score.
Suggesting that there exist large redundancies even in MNMT models. These
observations motivate us to use the redundant parameters and counter the
interference problem efficiently. We propose a novel adaptation strategy, where
we iteratively prune and retrain the redundant parameters of an MNMT to improve
bilingual representations while retaining the multilinguality. Negative
interference severely affects high resource languages, and our method
alleviates it without any additional adapter modules. Hence, we call it
parameter-free adaptation strategy, paving way for the efficient adaptation of
MNMT. We demonstrate the effectiveness of our method on a 9 language MNMT
trained on TED talks, and report an average improvement of +1.36 BLEU on high
resource pairs. Code will be released here.

Computation and Language