Neural inductive program synthesis is a task generating instructions that can produce desired outputs from given inputs. In this paper, we focus on the generation of a chunk of assembly code that can be executed to match a state change inside the CPU and RAM. We develop a neural program synthesis algorithm, AutoAssemblet, learned via self-learning reinforcement learning that explores the large code space efficiently. Policy networks and value networks are learned to reduce the breadth and depth of the Monte Carlo Tree Search, resulting in better synthesis performance. We also propose an effective multi-entropy policy sampling technique to alleviate online update correlations. We apply AutoAssemblet to basic programming tasks and show significant higher success rates compared to several competing baselines.
[Summary] 
This paper addresses the problem of synthesizing programs (x86 assembly code) from input/output (I/O) pairs. To this end, the paper proposes a framework (AutoAssemblet) that first learns a policy network and a value network using imitation learning (IL) and reinforcement learning (RL) and then leverages Monte Carlo Tree Search (MCTS) to infer programs. The experiments show that AutoAssemblet can synthesize assembly programs from I/O pairs to some extent. Ablation studies suggest the proposed IL and RL guided search is effective.
Significance: are the results significant? 2/5
Novelty: are the problems or approaches novel? 2/5
Evaluation: are claims well-supported by theoretical analysis or experimental results? 4/5
Clarity: is the paper well-organized and clearly written? 4/5
[Strengths]
*clarity*
The overall writing is clear. The authors utilize figures well to illustrate the ideas. Figure 1 clearly shows the proposed pipeline as well as the MCTS process. In general, the notations and formulations are well-explained. 
*technical contribution*
- Optimizing both the imitation learning loss and the reinforcement learning loss yields better performance when more tasks are available and tasks are more difficult.
- Leveraging a learned policy network and value network for improving the efficiency of the MCTS seems effective.
*ablation study*
Ablation studies are comprehensive. The proposed framework first optimizes two losses (IL and RL) and leverages the learned policy network and the value network for improving MCTS. The provided ablation studies help analyze the effectiveness of each of them.
*experimental results*
- All the descriptions of the experiments and the presentations of the results are fairly clear. 
- The results demonstrate the effectiveness of the proposed RL guided MCTS.
[Weaknesses]
*novelty*
Overall, I do not find enough novelty from any aspects while the overall effort of this paper is appreciated. The reasons are as follows.
- This  "self-learning" framework is not entirely novel since it has been proposed in [1], where the model is trained on a large number of programs that were randomly generated and tested on a real-world dataset (FlashFill). 
- The hybrid objective (IL+RL) has been explored in neural program synthesis [2] (a supervised learning model is fine-tuned using RL), learning robot manipulation [3], character control [4], etc.
- Utilizing Monte Carlo Tree Search for program synthesis has been studied in many works. [5] proposes to treat the network outputs as proposals for a sequential Monte Carlo sampling scheme and [6] presents an RL guided Monte Carlo tree search framework. 
- Program synthesis on assembly languages: RISC-V [6], etc.
*related work*
The descriptions of the related work are not comprehensive. While many neural synthesis works [1, 5, 7-10] have explored a wide range of settings for learning to perform program synthesis, they are not mentioned in the paper. I suggest the authors conduct a comprehensive survey on this line of works. 
*baselines*
In my opinion, the baselines (imitation, REINFORCE, MCTS) presented in the paper are far from comprehensive. I believe the following baselines should also be considered:
- As the proposed model optimizes a combination of the IL loss and the RL loss, it would make sense to also evaluate a model optimizing this hybrid loss.
- Search-based program synthesis baseline (i.e. learning guided search vs heuristic search)
- Comparing the proposed framework against some neural induction baselines would confirm the importance and effectiveness of explicitly synthesizing programs instead of directly predicting the outcome/output. This has been shown in [1, 7].
*testing set*
The testing sets are extremely small (with only 50, 40, 40 programs), which makes the results less convincing. Also, how those testing sets were created is not mentioned anywhere in the paper. It only states "we designed a set of human-designed tasks".
*number of given I/O pairs*
It is not mentioned anywhere how the authors split the observed I/O pairs and the assessment I/O pairs such as what has been done in most of the works [1, 2, 7, 8]. While the observed I/O pairs are input to the program synthesis framework, the assessment I/O pairs are used to evaluate the synthesized programs. By doing so, the more observed I/O pairs are given, the more accurate the synthesized programs should be (assuming the model can find programs that fit the observed I/O pairs). 
- The discovery (Figure 2d) in this paper is contradictory to what is mentioned above: the program synthesis accuracy decreases when more I/O pairs are given. I am assuming the authors do not split observed I/O pairs and assessment I/O pairs.
- With the setup where the observed I/O pairs are separated from the assessment I/O pairs, a larger number of K (the number of the observed I/O pairs) should be used so that it is more likely that the program for each task is unique and the evaluation would make more sense.
[1] "RobustFill: Neural Program Learning under Noisy I/O" in ICML 2017
[2] "Leveraging Grammar and Reinforcement Learning for Neural Program Synthesis" in ICLR 2018
[3] "Reinforcement and Imitation Learning for Diverse Visuomotor Skills" in RSS 2018
[4] "DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills" in SIGGRAPH 2018
[5] "Learning to Infer Graphics Programs from Hand-Drawn Images" in NeurIPS 2018
[6] "Program Synthesis Through Reinforcement Learning Guided Tree Search" arXiv 2018
[7] "Neural Program Synthesis from Diverse Demonstration Videos" in ICML 2019
[8] "Execution-Guided Neural Program Synthesis" in ICLR 2019
[9] "Learning to Describe Scenes with Programs" in ICLR 2019
[10] "Learning to Infer and Execute 3D Shape Programs" in ICLR 2019
===== After rebuttal =====
I appreciate the authors for the revision and for clarifying some points. I am still not entirely convinced by the response and the revision.
First, I mentioned several papers that are related to this work, the authors failed to discuss the difference between this submission and these works. 
- The "self-learning" paradigm is used in [1, 2, 7, 8] but [1, 7] are still not mentioned. If the authors intentionally ignore these works so that they can claim the novelty, it is not acceptable.
- While the proposed hybrid objective is very similar to the one proposed in [2, 3, 4], the revision does not mention this.
- Why [5, 6] that utilizing Monte Carlo Tree Search for program synthesis are still missing from the revision? I believe these works are very relevant to this submission.
- The authors acknowledged that they did not know about [6] that works on program synthesis for an assembly language. Yet, this paper is still missing from the paper.
I spent a lot of time conducting a survey in this field to help the authors to improve this submission and trying to identify the novelty of this submission. However,  the authors just chose to ignore it, which is very disappointing.
Second, many suggestions that I made are just rejected by the authors because they believe it is "not very easy and trivial". Given this self-learning setting, I believe it would be easy to implement some program induction baselines using supervised learning. Or how about search-based program synthesis baselines?
The writing about 50, 40, 40 testing sets is very misleading. I believe all the reviewers just think the testing accuracy was computed using those testing sets.
From my point of view, "program aliasing” does not mean there are not sufficient I/O examples to fully describe a task; Instead, it means there are programs written differently can be semantically the same, which means no matter how many input examples are given, their outputs will always match.
Overall, I believe this submission requires a serious revision and I firmly recommend this paper to be rejected.