Reasoning with Memory Augmented Neural Networks for Language Comprehension
Hypothesis testing is an important cognitive process that supports human reasoning. In this paper, we introduce a computational hypothesis testing approach based on memory augmented neural networks. Our approach involves a hypothesis testing loop that reconsiders and progressively refines a previously formed hypothesis in order to generate new hypotheses to test. We apply the proposed approach to language comprehension task by using Neural Semantic Encoders (NSE). Our NSE models achieve the state-of-the-art results showing an absolute improvement of 1.2% to 2.6% accuracy over previous results obtained by single and ensemble systems on standard machine comprehension benchmarks such as the Children*s Book Test (CBT) and Who-Did-What (WDW) news article datasets.
First I would like to apologize for the delay in reviewing. Summary : this work introduces a novel memory based artificial neural network for reading comprehension. Experiments show improvement on state of the art. The originality of the approach seems to be on the implementation of an iterative procedure with a loop testing that the current answer is the correct one. In order to get a better sense of the reason for improvement it would be interesting to have a complexity and/or a time analysis of the algorithm. I might be mistaken but I don*t see you reporting anything on the actual number of loops necessary in the reported experiments. The dataset description in section 2.2, should be moved to section 4 where the other datasets are described.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Reasoning with Memory Augmented Neural Networks for Language Comprehension
Hypothesis testing is an important cognitive process that supports human reasoning. In this paper, we introduce a computational hypothesis testing approach based on memory augmented neural networks. Our approach involves a hypothesis testing loop that reconsiders and progressively refines a previously formed hypothesis in order to generate new hypotheses to test. We apply the proposed approach to language comprehension task by using Neural Semantic Encoders (NSE). Our NSE models achieve the state-of-the-art results showing an absolute improvement of 1.2% to 2.6% accuracy over previous results obtained by single and ensemble systems on standard machine comprehension benchmarks such as the Children*s Book Test (CBT) and Who-Did-What (WDW) news article datasets.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



First I would like to apologize for the delay in reviewing. Summary : this work introduces a novel memory based artificial neural network for reading comprehension. Experiments show improvement on state of the art. The originality of the approach seems to be on the implementation of an iterative procedure with a loop testing that the current answer is the correct one. In order to get a better sense of the reason for improvement it would be interesting to have a complexity and/or a time analysis of the algorithm. I might be mistaken but I don*t see you reporting anything on the actual number of loops necessary in the reported experiments. The dataset description in section 2.2, should be moved to section 4 where the other datasets are described.
assistant
First I would like to apologize for the delay in reviewing. Summary : this work introduces a novel memory based artificial neural network for reading comprehension. Experiments show improvement on state of the art. The originality of the approach seems to be on the implementation of an iterative procedure with a loop testing that the current answer is the correct one. In order to get a better sense of the reason for improvement it would be interesting to have a complexity and/or a time analysis of the algorithm. I might be mistaken but I don*t see you reporting anything on the actual number of loops necessary in the reported experiments. The dataset description in section 2.2, should be moved to section 4 where the other datasets are described.