Many challenging prediction problems, from molecular optimization to program synthesis, involve creating complex structured objects as outputs. However, available training data may not be sufficient for a generative model to learn all possible complex transformations. By leveraging the idea that evaluation is easier than generation, we show how a simple, broadly applicable, iterative target augmentation scheme can be surprisingly effective in guiding the training and use of such models. Our scheme views the generative model as a prior distribution, and employs a separately trained filter as the likelihood. In each augmentation step, we filter the model's outputs to obtain additional prediction targets for the next training epoch. Our method is applicable in the supervised as well as semi-supervised settings. We demonstrate that our approach yields significant gains over strong baselines both in molecular optimization and program synthesis. In particular, our augmented model outperforms the previous state-of-the-art in molecular optimization by over 10% in absolute gain.
(post rebuttal) I appreciate the authors for detailed rebuttal. I stay with the original score based on the following reason.
In RAML you sample from exponentiated reward distribution and do maximum likelihood (minimizing forward KL in classic control as inference framework). How you sample depends on the problem assumption. In the original paper, they used a heuristic based on edit distance from ground truth, but in theirs they did not have assumption that you can evaluate correctness (as used in external filter). In this paper's context, the sampling naturally comes down to rejection sampling. Therefore I keep the original stance regarding the similarity between this work and RAML. 
---
The paper proposes an iterative data augmentation approach based self-generation and filtering with success criteria. The authors justify the algorithm as an EM procedure of maximizing \log p^*(y|x), where p^*(y|x) \propto p(y|x) * p(c=1|x,y). They demonstrate that the iterative data augmentation procedure can provide significant gains to SOA models in molecule generation and outperform MLE+RL method in program synthesis datasets.
Strengths:
- A simple approach that can be added to any seq2seq translation where success metric can be evaluated efficiently
- Demonstrated results on multiple applications
Weaknesses:
- The method requires being able to evaluate constraints  
- The approach is simple and does not seem to present significant novelty over prior methods. Particularly, the approach could be considered as nesting RAML [1] updates with (1) low temperature, (2) self-generated trajectories. 
Other comments:
- Some missing references [1, 2]. [2] also studies molecule generation and proposes approaches similar to RL + MLE. 
[1] Norouzi, Mohammad, et al. "Reward augmented maximum likelihood for neural structured prediction." Advances In Neural Information Processing Systems. 2016.
[2] Jaques, Natasha, et al. "Sequence tutor: Conservative fine-tuning of sequence generation models with kl-control." Proceedings of the 34th International Conference on Machine Learning-Volume 70. JMLR. org, 2017.