People ask questions that are far richer, more informative, and more creative than current AI systems. We propose a neural program generation framework for modeling human question asking, which represents questions as formal programs and generates programs with an encoder-decoder based deep neural network. From extensive experiments using an information-search game, we show that our method can ask optimal questions in synthetic settings, and predict which questions humans are likely to ask in unconstrained settings. We also propose a novel grammar-based question generation framework trained with reinforcement learning, which is able to generate creative questions without supervised data.
[Summary] 
This paper studies a very interesting problem - if machines can learn to ask the right questions to address or gain crucial information about tasks. I believe this is a very important yet under-explored problem. Specifically, this paper considers a setting where a neural network model is trained to ask questions by predicting a formal program. The network consists of a CNN encoder which encodes a partially observable state and a Transformer decoder that generates a program as a sequence of tokens. The experiments on a Battleship task show some promising results. 
Significance: are the results significant? 3/5
Novelty: are the problems or approaches novel? 3/5
Evaluation: are claims well-supported by theoretical analysis or experimental results? 4/5
Clarity: is the paper well-organized and clearly written? 4.5/5
[Strengths]
*motivation*
The motivation for investigating machines' ability to generating questions to gain important information is convincing.
*novelty*
The idea of utilizing learning models for generating questions and modeling human-generated questions intuitive and convincing. This paper presents an effective way to implement this idea.
*technical contribution*
Leveraging a context-free grammar (CFG) and reinforcement learning (RL) for question generation seems effective especially when it comes to generating unique and novel questions.
*clarity*
The overall writing is clear and the authors utilize figures well to illustrate the ideas. Figure 1 illustrates the Battleship task and Figure 2 presents a clear overview of the proposed framework.
*ablation study*
Ablation studies are comprehensive. The proposed framework consists of multiple components. The provided ablation studies (Table 2 and Table 4) help analyze the effectiveness of each of them.
*experimental results*
- The presentations of the results are clear. The conclusions are fairly convincing: 
- Experiment 1: learning to generate programmatic questions encourages learning rules for composition
- Experiment 2: it is possible to capture the human-generated questions distribution as a conditioned language model.
- Experiment 3: CFG+RL can produce more diverse and novel questions compared to supervised learning or RL.
[Weaknesses]
*novelty & contribution*
Overall, I do not find enough novelty from any aspects while the overall effort of this paper is appreciated. 
- The problem is not entirely novel as [1-3] have explored asking questions with neural networks learning to generate programmatic questions.
- The proposed framework leverages a CFG and learns with RL, while the former (program synthesis with a CFG) has been studied in [4-5] and the latter (program synthesis using RL) has been discussed in [4, 6-7].
- Many setups considered in this paper have been explored in several neural program synthesis papers [8-12], which are neglected from this paper.
- The conclusions presented in the experiment section are more or less expected (i.e. other works have presented similar results on slightly different problems).
*oversell the motivation*
I believe the authors oversell the key motivation (i.e. to enable learning agents to ask questions) a little bit. To be more specific, it would be more interesting if an agent is allowed to ask a question, take actions based on the answer, and then ask the next question. However, this paper investigates the case where a significant amount of human-generated questions are given, which could limit the learning in my opinion. Also, it is not clear to me how the setup evaluated in this paper can be extended to a setting that allows an agent to iteratively ask questions.
*experiment setup*
Do the models learning using RL get the reward based on just the ground truth questions (a sequence of tokens) or the execution results? 
*format*
The title & reference format looks wrong
*reference*
[1] Question Asking as Program Generation in NIPS 2017
[2] Neural-Symbolic VQA: Disentangling Reasoning from Vision and Language Understanding in NeurIPS 2018
[3] The Neuro-Symbolic Concept Learner: Interpreting Scenes, Words, and Sentences From Natural Supervision in ICLR 2019
[4] Leveraging grammar and reinforcement learning for neural program synthesis in ICLR 2018
[5] Learning a Meta-Solver for Syntax-Guided Program Synthesis in ICLR 2019
[6] Neural Scene De-rendering in CVPR 2017
[7] Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning
[8] RobustFill: Neural Program Learning under Noisy I/O in ICML 2017
[9] Execution-Guided Neural Program Synthesis in ICLR 2019
[10] Neural Program Synthesis from Diverse Demonstration Videos  in ICML 2018
[11] Learning to Describe Scenes with Programs in ICLR 2019
[12] Learning to Infer and Execute 3D Shape Programs in ICLR 2019