Published as a conference paper at ICLR 2018 LEARNING TO REPRESENT PROGRAMS WITH GRAPHS
Learning tasks on source code (i.e., formal languages) have been considered recently, but most work has tried to transfer natural language methods and does not capitalize on the unique opportunities offered by code's known sematics. For example, long-range dependencies induced by using the same variable or function in distant locations are often not considered. We propose to use graphs to represent both the syntactic and semantic structure of code and use graph-based deep learning methods to learn to reason over program structures. In this work, we present how to construct graphs from source code and how to scale Gated Graph Neural Networks training to such large graphs. We evaluate our method on two tasks: VARNAMING, in which a network attempts to predict the name of a variable given its usage, and VARMISUSE, in which the network learns to reason about selecting the correct variable that should be used at a given program location. Our comparison to methods that use less structured program representations shows the advantages of modeling known structure, and suggests that our models learn to infer meaningful names and to solve the VARMISUSE task in many cases. Additionally, our testing showed that VARMISUSE identifies a number of bugs in mature open-source projects.
Summary:  The paper applies graph convolutions with deep neural networks to the problem of "variable misuse" (putting the wrong variable name in a program statement) in graphs created deterministically from source code.  Graph structure is determined by program abstract syntax tree (AST) and next-token edges, as well as variable/function name identity, assignment and other deterministic semantic relations.  Initial node embedding comes from both type and tokenized name information.  Gated Graph Neural Networks (GGNNs, trained by maximum likelihood objective) are then run for 8 iterations at test time.
The evaluation is extensive and mostly very good.  Substantial data set of 29m lines of code.  Reasonable baselines.  Nice ablation studies.  I would have liked to see separate precision and recall rather than accuracy.  The current 82.1% accuracy is nice to see, but if 18% of my program variables were erroneously flagged as errors, the tool would be useless.  I'd like to know if you can tune the threshold to get a precision/recall tradeoff that has very few false warnings, but still catches some errors.
Nice work creating an implementation of fast GGNNs with large diverse graphs.  Glad to see that the code will be released.  Great to see that the method is fast---it seems fast enough to use in practice in a real IDE.
The model (GGNN) is not particularly novel, but I'm not much bothered by that.  I'm very happy to see good application papers at ICLR.  I agree with your pair of sentences in the conclusion: "Although source code is well understood and studied within other disciplines such as programming language research, it is a relatively new domain for deep learning. It presents novel opportunities compared to textual or perceptual data, as its (local) semantics are well-defined and rich additional information can be extracted using well-known, efficient program analyses."  I'd like to see work in this area encouraged.  So I recommend acceptance.  If it had better (e.g. ROC curve) evaluation and some modeling novelty, I would rate it higher still.
Small notes:
The paper uses the term "data flow structure" without defining it.
Your data set consisted of C# code.  Perhaps future work will see if the results are much different in other languages.