Emergent Predication Structure in Vector Representations of Neural Readers
Reading comprehension is a question answering task where the answer is to be found in a given passage about entities and events not mentioned in general knowledge sources. A significant number of neural architectures for this task (neural readers) have recently been developed and evaluated on large cloze-style datasets. We present experiments supporting the emergence of “predication structure” in the hidden state vectors of a class of neural readers including the Attentive Reader and Stanford Reader. We posits that the hidden state vectors can be viewed as (a representation of) a concatenation [P, c] of a “predicate vector” P and a “constant symbol vector” c and that the hidden state represents the atomic formula P(c). This predication structure plays a conceptual role in relating “aggregation readers” such as the Attentive Reader and the Stanford Reader to “explicit reference readers” such as the Attention-Sum Reader, the Gated-Attention Reader and the Attention-over-Attention Reader. In an independent contribution, we show that the addition of linguistics features to the input to existing neural readers significantly boosts performance yielding the best results to date on the Who-did-What dataset.
This paper aims to provide an insightful and analytic survey over the recent literature on reading comprehension with the distinct goal of investigating whether logical structure (or predication, as the authors rephrased in their response) arises in many of the recent models. I really like the spirit of the paper and appreciate the efforts to organize rather chaotic recent literature into two unified themes: *aggregation readers* and *explicit reference models”. Overall the quality of writing is great and section 3 was especially nice to read. I’m also happy with the proposed rewording from *logical structure* to “predication*, and the clarification by the authors was detailed and helpful. I think I still have slight mixed feelings about the contribution of the work. First, I wonder whether the choice of the dataset was ideal in the first place to accomplish the desired goal of the paper. There have been concerns about CNN/DailyMail dataset (Chen et al. ACL’16) and it is not clear to me whether the dataset supports investigation on logical structure of interesting kinds. Maybe it is bound to be rather about lack of logical structure. Second, I wish the discussion on predication sheds more practical insights into dataset design or model design to better tackle reading comprehension challenges. In that sense, it may have been more helpful if the authors could make more precise analysis on different types of reading comprehension challenges, what types of logical structure are lacking in various existing models and datasets, and point to specific directions where the community needs to focus more.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Emergent Predication Structure in Vector Representations of Neural Readers
Reading comprehension is a question answering task where the answer is to be found in a given passage about entities and events not mentioned in general knowledge sources. A significant number of neural architectures for this task (neural readers) have recently been developed and evaluated on large cloze-style datasets. We present experiments supporting the emergence of “predication structure” in the hidden state vectors of a class of neural readers including the Attentive Reader and Stanford Reader. We posits that the hidden state vectors can be viewed as (a representation of) a concatenation [P, c] of a “predicate vector” P and a “constant symbol vector” c and that the hidden state represents the atomic formula P(c). This predication structure plays a conceptual role in relating “aggregation readers” such as the Attentive Reader and the Stanford Reader to “explicit reference readers” such as the Attention-Sum Reader, the Gated-Attention Reader and the Attention-over-Attention Reader. In an independent contribution, we show that the addition of linguistics features to the input to existing neural readers significantly boosts performance yielding the best results to date on the Who-did-What dataset.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



This paper aims to provide an insightful and analytic survey over the recent literature on reading comprehension with the distinct goal of investigating whether logical structure (or predication, as the authors rephrased in their response) arises in many of the recent models. I really like the spirit of the paper and appreciate the efforts to organize rather chaotic recent literature into two unified themes: *aggregation readers* and *explicit reference models”. Overall the quality of writing is great and section 3 was especially nice to read. I’m also happy with the proposed rewording from *logical structure* to “predication*, and the clarification by the authors was detailed and helpful. I think I still have slight mixed feelings about the contribution of the work. First, I wonder whether the choice of the dataset was ideal in the first place to accomplish the desired goal of the paper. There have been concerns about CNN/DailyMail dataset (Chen et al. ACL’16) and it is not clear to me whether the dataset supports investigation on logical structure of interesting kinds. Maybe it is bound to be rather about lack of logical structure. Second, I wish the discussion on predication sheds more practical insights into dataset design or model design to better tackle reading comprehension challenges. In that sense, it may have been more helpful if the authors could make more precise analysis on different types of reading comprehension challenges, what types of logical structure are lacking in various existing models and datasets, and point to specific directions where the community needs to focus more.
assistant
This paper aims to provide an insightful and analytic survey over the recent literature on reading comprehension with the distinct goal of investigating whether logical structure (or predication, as the authors rephrased in their response) arises in many of the recent models. I really like the spirit of the paper and appreciate the efforts to organize rather chaotic recent literature into two unified themes: *aggregation readers* and *explicit reference models”. Overall the quality of writing is great and section 3 was especially nice to read. I’m also happy with the proposed rewording from *logical structure* to “predication*, and the clarification by the authors was detailed and helpful. I think I still have slight mixed feelings about the contribution of the work. First, I wonder whether the choice of the dataset was ideal in the first place to accomplish the desired goal of the paper. There have been concerns about CNN/DailyMail dataset (Chen et al. ACL’16) and it is not clear to me whether the dataset supports investigation on logical structure of interesting kinds. Maybe it is bound to be rather about lack of logical structure. Second, I wish the discussion on predication sheds more practical insights into dataset design or model design to better tackle reading comprehension challenges. In that sense, it may have been more helpful if the authors could make more precise analysis on different types of reading comprehension challenges, what types of logical structure are lacking in various existing models and datasets, and point to specific directions where the community needs to focus more.