Published as a conference paper at ICLR 2018 SYNTAX-DIRECTED VARIATIONAL AUTOENCODER FOR STRUCTURED DATA
Deep generative models have been enjoying success in modeling continuous data. However it remains challenging to capture the representations for discrete structures with formal grammars and semantics, e.g., computer programs and molecular structures. How to generate both syntactically and semantically correct data still remains largely an open problem. Inspired by the theory of compiler where the syntax and semantics check is done via syntax-directed translation (SDT), we propose a novel syntax-directed variational autoencoder (SD-VAE) by introducing stochastic lazy attributes. This approach converts the offline SDT check into on-the-fly generated guidance for constraining the decoder. Comparing to the state-of-the-art methods, our approach enforces constraints on the output space so that the output will be not only syntactically valid, but also semantically reasonable. We evaluate the proposed model with applications in programming language and molecules, including reconstruction and program/molecule optimization. The results demonstrate the effectiveness in incorporating syntactic and semantic constraints in discrete generative models, which is significantly better than current state-of-the-art approaches.
NOTE: 
Would the authors kindly respond to the comment below regarding Kekulisation of the Zinc dataset? Fair comparison of the data is a serious concern. I have listed this review as a good for publication due to the novelty of ideas presented, but the accusation of misrepresentation below is a serious one and I would like to know the author's response.
*Overview*
This paper presents a method of generating both syntactically and semantically valid data from a variational autoencoder model using ideas inspired by compiler semantic checking. Instead of verifying the semantic correctness offline of a particular discrete structure, the authors propose “stochastic lazy attributes”, which amounts to loading semantic constraints into a CFG and using a tailored latent-space decoder algorithm that guarantees both syntactic semantic valid. Using Bayesian Optimization, search over this space can yield decodings with targeted properties.
Many of the ideas presented are novel. The results presented are state-of-the art. As noted in the paper, the generation of syntactically and semantically valid data is still an open problem. This paper presents an interesting and valuable solution, and as such constitutes a large advance in this nascent area of machine learning.
*Remarks on methodology*
By initializing a decoding by “guessing” a value, the decoder will focus on high-probability starting regions of the space of possible structures. It is not clear to me immediately how this will affect the output distribution. Since this process on average begins at high-probability region and makes further decoding decisions from that starting point, the output distribution may be biased since it is the output of cuts through high-probability regions of the possible outputs space. Does this sacrifice exploration for exploitation in some quantifiable way? Some exploration of this issue or commentary would be valuable. 
*Nitpicks*
I found the notion of stochastic predetermination somewhat opaque, and section 3 in general introduces much terminology, like lazy linking, that was new to me coming from a machine learning background. In my opinion, this section could benefit from a little more expansion and conceptual definition.
The first 3 sections of the paper are very clearly written, but the remainder has many typos and grammatical errors (often word omission). The draft could use a few more passes before publication.