MULTI-AGENT DUAL LEARNING
Dual learning has attracted much attention in machine learning, computer vision and natural language processing communities. The core idea of dual learning is to leverage the duality between the primal task (mapping from domain X to domain Y) and dual task (mapping from domain Y to X ) to boost the performances of both tasks. Existing dual learning framework forms a system with two agents (one primal model and one dual model) to utilize such duality. In this paper, we extend this framework by introducing multiple primal and dual models, and propose the multi-agent dual learning framework. Experiments on neural machine translation and image translation tasks demonstrate the effectiveness of the new framework. In particular, we set a new record on IWSLT 2014 German-to-English translation with a 35.44 BLEU score, achieve a 31.03 BLEU score on WMT 2014 English-toGerman translation with over 2.6 BLEU improvement over the strong Transformer baseline, and set a new record of 49.61 BLEU score on the recent WMT 2018 English-to-German translation.
Summary
The paper proposes to modify the "Dual Learning" approach to supervised (and unsupervised) translation problems by making use of additional pretrained mappings for both directions (i.e. primal and dual). These pre-trained mappings ("agents") generate targets from the primal to the dual domain, which need to be mapped back to the original input. It is shown that having >=1 additional agents improves training of the BLEU score in standard MT and unsupervised MT tasks. The method is also applied to unsupervised image-to-image "translation" tasks.
Positives and Negatives
+1 Simple and straightforward method with pretty good results on language translation.
+2 Does not require additional computation during inference, unlike ensembling.
-1 The mathematics in section 3.1 is unclear and potentially flawed (more below).
-2 Diversity of additional "agents" not analyzed (more below).
-3 For image-to-image translation experiments, no quantitative analysis whatsoever is offered so the reader can't really conclude anything about the effect of the proposed method in this domain.
-4 Talking about "agents" and "Multi-Agent" is a somewhat confusing given the slightly different use of the same term in the reinforcement literature. Why not just "mapping" or "network"?
-1: Potential Issues with the Maths.
The maths is not clear, in particular the gradient derivation in equation (8). Let's just consider the distortion objective on x (of course it also applies to y without loss of generality). At the very least we need another "partial" sign in front of the "\delta" function in the numerator. But again, it's not super clear how the paper estimates this derivative.  Intuitively the objective wants f_0 to generate samples which, when mapped back to the X domain, have high log-probability under G, but its samples cannot be differentiated in the case of discrete data. So is the REINFORCE estimator used or something? Not that the importance sampling matter is orthogonal. In the case of continuous data x, is the reparameterization trick used? This should at the very least be explained more clearly.
Note that the importance sampling does not affect this issue.
-2: Diversity of Agents.
As with ensembles, clearly it only helps to have multiple agents (N>2) if the additional agents are distinct from f_1 (again without loss of generality this applies to g as well). The paper proposes to use different random seeds and iterate over the dataset in a different order for distinct pretrained f_i. The paper should quantify that this leads to diverse "agents". I suppose the proof is in the pudding; as we have argued, multiple agents can only improve performance if they are distinct, and Figure 1 shows some improvement as the number of agents are increase (no error bars though). The biggest jump seems to come from N=1 -> N=2 (although N=4 -> N=5 does see a jump as well). Presumably if you get a more diverse pool of agents, that should improve things. Have you considered training different agents on different subsets of the data, or trying different learning algorithms/architectures to learn them? More experiments on the diversity would help make the paper more convincing.