As for knowledge-based question answering, a fundamental problem is to relax the assumption of answerable questions from simple questions to compound questions. Traditional approaches firstly detect topic entity mentioned in questions, then traverse the knowledge graph to find relations as a multi-hop path to answers, while we propose a novel approach to leverage simple-question answerers to answer compound questions. Our model consists of two parts: (i) a novel learning-to-decompose agent that learns a policy to decompose a compound question into simple questions and (ii) three independent simple-question answerers that classify the corresponding relations for each simple question. Experiments demonstrate that our model learns complex rules of compositionality as stochastic policy, which benefits simple neural networks to achieve state-of-the-art results on WebQuestions and MetaQA. We analyze the interpretable decomposition process as well as generated partitions.
Summary: the paper is interested in parsing compound questions for querying on knowledge graph, e.g. MetaQA by Zhang et al. (2017). The paper proposes to have two modules, one that segments the question into partitions (up to three) and the other that looks at each segment to get the relation. The relations are merged to obtain a single KG path, which is queried to obtain the answer. Since the segmentation is a non-differentiable process, the paper uses reinforcement learning to propagate gradient to the segmentation model. The segmentation is a process of classifying each word for which partition it should be tied to. Answering is a process of classifying the partition into one of the possible relation edges. The model shows expected results in a synthetic arithmetic dataset, and obtains the state of the art in MetaQA, improving nearly 5% over the baseline. The model especially does much better on 3-hop questions, with nearly 20% improvement.
Strengths: the paper is well-written. The model is simple yet effective and is a novel contribution to compound question answering on KG. Especially, the improvement on 3-hop category is nearly 20%, which is substantial and quite impressive. 
Weaknesses: My biggest concern is the lack of discussions on its relevance to  (Iyyer et al., 2016), which also proposed to decompose question into simpler ones for WIkiTableQuestions. Also, I think it would be good to mention Semantic Role Labeling as related literature, which is about tagging each word with its role in the sentence. The partition index can be somewhat considered as a “role” in the sentence.
Questions:
1. How do you obtain x^(k)? Is it the last state of the LSTM?
2. Why did you have to augment “NO_OP” relation in the MetaQA dataset?
3. Why +1 reward has lower variance than probabilistic reward? Explanation or citation would be needed.
4. What if two partitions need to share a word? The current setup necessitates that a word participates in only one partition. Wouldn’t this be problematic?
5. I am a bit confused about how the simple question answering module is trained. Is it directly trained by the gold relation label?
Typos and Suggestions:
- Second paragraph of 2.1: in stead -> instead
- Third paragraph of 2.1: research. -> research
- c_t + h_t: would be good to explicitly mention that the circled plus sign is concatenation.
- Last paragraph on page 4: “leave to be”?
- Second last paragraph of 4.1: he -> The
- Second paragraph of 4.2: “if exists a proper meaning”?
- First paragraph of page 7: be either assume -> either assume
- Last paragraph of Section 5: generalizing -> generalize
- I think you should not put acknowledgment in a double-blind submission.
M Iyyer, W Yih, MW Chang. Answering complicated question intents expressed in decomposed question sequences. 2016 (https://arxiv.org/abs/1611.01242)