Neural Machine Translation (NMT) systems rely on large amounts of parallel data. This is a major challenge for low-resource languages. Building on recent work on unsupervised and semi-supervised methods, we present an approach that combines zero-shot and dual learning. The latter relies on reinforcement learning, to exploit the duality of the machine translation task, and requires only monolingual data for the target language pair. Experiments on the UN corpus show that a zero-shot dual system, trained on English-French and English-Spanish, outperforms by large margins a standard NMT system in zero-shot translation performance on SpanishFrench (both directions). We also evaluate on newstest2014. These experiments show that the zero-shot dual method outperforms the LSTM-based unsupervised NMT system proposed in (Lample et al., 2018b), on the en− →fr task, while on the fr− →en task it outperforms both the LSTM-based and the Transformers-based unsupervised NMT systems.
Pros:
- The paper address the problem of zero-shot translation. The proposed method is essentially to bootstrap a Dual Learning process using a multilingual translation model that already has some degree of zero-shot translation capabilities. The idea is simple, but the approach improves the zero-shot translation performance of the baseline model, and seems to be better than either pivoting or training on direct but out-of-domain parallel data.
- The paper is mostly well written and easy to follow. There are some missing details that I've listed below.
Cons:
- There is very little comparison to related work. For example, related work by Chen et al. [1], Gu et al. [2] and Lu et al. [3] are not cited nor compared against.
Misc questions/comments:
- In a few places you call your approach unsupervised (e.g., in Section 3: "Our method for unsupervised machine translation works as follows: (...)"; Section 5.2 is named "Unsupervised Performance"). But your method is not unsupervised in the traditional sense, since you require lots of parallel data for the target languages, just not necessarily directly between the pair. This may be unrealistic in low-resource settings if there is not an existing suitable pivot language. It'd be more accurate to simply say "zero-shot" (or maybe "semi-supervised") in Section 3 and Section 5.2.
- In Section 3.1 you say that your process implements the three principles outlined in Lample et al. (2018b). However, the Initialization principle in that work refers to initializing the embeddings -- do you pretrain the word embeddings as well?
- In Section 4 you say that the "UN corpus is of sufficient size". Please mention what the size is.
- In Section 4.2, you mention that you set dropout to p=0.65 when training your language model -- this is very high! Did you tune this? Does your language model overfit very badly with lower dropout values?
- In Section 5.2, what is the BLEU of an NMT system trained on the es->fr data (i.e., what is the upper bound)? What is the performance of a pivoting model?
- In Section 5.3, you say you use "WMT News Crawl, all years." Please indicate which years explicitly.
- In Table 3, what is the performance of a supervised NMT system trained on 1M en-fr sentences of the NC data? Knowing that would help clarify the impact of the domain mismatch.
- minor comment: in Section 4.3 you say that you trained on Tesla-P100, but do you mean Pascal P100 or Tesla V100?
[1] Chen et al.: http://aclweb.org/anthology/P17-1176
[2] Gu et al.: http://aclweb.org/anthology/N18-1032
[3] Lu et al.: http://www.statmt.org/wmt18/pdf/WMT009.pdf