Workshop track -ICLR 2018 TREE-TO-TREE NEURAL NETWORKS FOR PROGRAM TRANSLATION
Program translation is an important tool to migrate legacy code in one language into an ecosystem built in a different language. In this work, we are the first to consider employing deep neural networks toward tackling this problem. We observe that program translation is a modular procedure, in which a sub-tree of the source tree is translated into the corresponding target sub-tree at each step. To capture this intuition, we design a tree-to-tree neural network as an encoderdecoder architecture to translate a source tree into a target one. Meanwhile, we develop an attention mechanism for the tree-to-tree model, so that when the decoder expands one non-terminal in the target tree, the attention mechanism locates the corresponding sub-tree in the source tree to guide the expansion of the decoder. We evaluate the program translation capability of our tree-to-tree model against several state-of-the-art approaches. Compared against other neural translation models, we observe that our approach is consistently better than the baselines with a margin of up to 15 points. Further, our approach can improve the previous state-of-the-art program translation approaches by a margin of 20 points on the translation of real-world projects.
This paper aims to translate source code from one programming language to another using
a neural network architecture that maps trees to trees. The encoder uses an upward pass of
a Tree LSTM to compute embeddings for each subtree of the input, and then the decoder 
constructs a tree top-down. As nodes are created in the decoder, a hidden state is passed
from parents to children via an LSTM (one for left children, one for right children), and
an attention mechanism allows nodes in the decoder to attend to subtrees in the encoder.
Experimentally, the model is applied to two synthetic datasets, where programs in the 
source domain are sampled from a PCFG and then translated to the target domain with a
hand-coded translator. The model is then trained on these pairs. Results show that the
nproposed approach outperforms sequence representations or serialized tree representations
of inputs and outputs.
Pros:
- Nice model which seems to perform well.
- Reasonably clear explanation.
A couple questions about the model:
- the encoder uses only bottom-up information to determine embeddings of subtrees. I wonder 
if top-down information would create embeddings with more useful information for the attention
in the decoder to pick up on.
- I would be interested to know more details about how the hand-coded translator works. Does
it work in a context-free, bottom-up fashion? That is, recursively translate two children nodes
and then compute the translation of the parent as a function of the parent node and
translations of the two children? If so, I wonder what is missing from the proposed model
that makes it unable to perfectly solve the first task?
Cons:
- Only evaluated on synthetic programs, and PCFGs are known to generate unrealistic programs, 
so we can only draw limited conclusions from the results.
- The paper overstates its novelty and doesn't properly deal with related work (see below)
The paper overstates its novelty and has done a poor job researching related work. 
Statements like "We are the first to consider employing neural network approaches 
towards tackling the problem [of translating between programming languages]" are
obviously not true (surely many people have *considered* it), and they're particularly
grating when the treatment of related  work is poor, as it is in this paper. For example, 
there are several papers that frame the code migration problem as one of statistical 
machine translation (see Sec 4.4 of [1] for a review and citations), but this paper 
makes no reference to them. Further, [2] uses distributed representations for the purpose 
of code migration, which I would call a "neural network approach," so there's not any 
sense that I can see in which this statement is true. The paper further says, "To the best 
of our knowledge, this is the first tree-to-tree neural network architecture in the 
literature." This is worded better, but it's definitely not the first tree-to-tree 
neural network. See, e.g., [3, 4, 5], one of which is cited, so I'm confused about 
this claim.
In total, the model seems clean and somewhat novel, but it has only been tested on 
unrealistic synthetic data, the framing with respect to related work is poor, and the
contributions are overstated.
[1] https://arxiv.org/abs/1709.06182
[2]  Trong Duc Nguyen, Anh Tuan Nguyen, and Tien N Nguyen. 2016b. Mapping API elements for code migration with
vector representations. In Proceedings of the International Conference on Software Engineering (ICSE).
[3] Socher, Richard, et al. "Semi-supervised recursive autoencoders for predicting sentiment distributions." Proceedings of the conference on empirical methods in natural language processing. Association for Computational Linguistics, 2011.
[4] https://arxiv.org/abs/1703.01925
[5] Parisotto, Emilio, et al. "Neuro-symbolic program synthesis." arXiv preprint arXiv:1611.01855 (2016).