Natural Language Inference over Interaction Space
Natural Language Inference (NLI) task requires an agent to determine the logical relationship between a natural language premise and a natural language hypothesis. We introduce Interactive Inference Network (IIN), a novel class of neural network architectures that is able to achieve high-level understanding of the sentence pair by hierarchically extracting semantic features from interaction space. We show that an interaction tensor (attention weight) contains semantic information to solve natural language inference, and a denser interaction tensor contains richer semantic information. One instance of such architecture, Densely Interactive Inference Network (DIIN), demonstrates the state-of-the-art performance on large scale NLI copora and large-scale NLI alike corpus. It*s noteworthy that DIIN achieve a greater than 20% error reduction on the challenging Multi-Genre NLI (MultiNLI) dataset with respect to the strongest published system.
 Pros: The paper proposes a “Densely Interactive Inference Network (DIIN)” for NLI or NLI alike tasks. Although using tensors to capture high-order interaction and performing dimension reduction over that are both not novel, the paper explores them for NLI. The paper is written clearly and is very easy to follow. The ablation experiments in Table 5 give a good level of details to help observe different components* effectiveness. Cons: 1) The differences of performances between the proposed model and the previous models are not very clear. With regard to MultiNLI, since the previous results (e.g., those in Table 2) did not use cross-sentence attention and had to represent a premise or a hypothesis as a *fixed-length* vector, is it fair to compare DIIN with them? Note that the proposed DIIN model does represent a premise or a hypothesis by variable lengths (see interaction layer in Figure 1), and tensors provide some sorts of attention between them. Can this (Table 2) really shows the advantage of the proposed models? However, when a variable-length representation is allowed (see Table 3 on SNLI), the advantage of the model is also not observed, with no improvement as a single model (compared with ESIM) and being almost same as previous models (e.g., model 18 in Table 3) in ensembling. 2) Method-wise, as discussed above, using tensors to capture high-order interaction and performing dimension reduction over that are both not novel. 3) The paper mentions the use of untied parameters for premise and hypothesis, but it doesn’t compare it with tied version in the experiment section. 4) In Table 6, for CONDITIONAL tag, why the baseline models (lower total accuracies) have a 100% accuracy, but DIIN only has about a 60% accuracy?
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Natural Language Inference over Interaction Space
Natural Language Inference (NLI) task requires an agent to determine the logical relationship between a natural language premise and a natural language hypothesis. We introduce Interactive Inference Network (IIN), a novel class of neural network architectures that is able to achieve high-level understanding of the sentence pair by hierarchically extracting semantic features from interaction space. We show that an interaction tensor (attention weight) contains semantic information to solve natural language inference, and a denser interaction tensor contains richer semantic information. One instance of such architecture, Densely Interactive Inference Network (DIIN), demonstrates the state-of-the-art performance on large scale NLI copora and large-scale NLI alike corpus. It*s noteworthy that DIIN achieve a greater than 20% error reduction on the challenging Multi-Genre NLI (MultiNLI) dataset with respect to the strongest published system.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



 Pros: The paper proposes a “Densely Interactive Inference Network (DIIN)” for NLI or NLI alike tasks. Although using tensors to capture high-order interaction and performing dimension reduction over that are both not novel, the paper explores them for NLI. The paper is written clearly and is very easy to follow. The ablation experiments in Table 5 give a good level of details to help observe different components* effectiveness. Cons: 1) The differences of performances between the proposed model and the previous models are not very clear. With regard to MultiNLI, since the previous results (e.g., those in Table 2) did not use cross-sentence attention and had to represent a premise or a hypothesis as a *fixed-length* vector, is it fair to compare DIIN with them? Note that the proposed DIIN model does represent a premise or a hypothesis by variable lengths (see interaction layer in Figure 1), and tensors provide some sorts of attention between them. Can this (Table 2) really shows the advantage of the proposed models? However, when a variable-length representation is allowed (see Table 3 on SNLI), the advantage of the model is also not observed, with no improvement as a single model (compared with ESIM) and being almost same as previous models (e.g., model 18 in Table 3) in ensembling. 2) Method-wise, as discussed above, using tensors to capture high-order interaction and performing dimension reduction over that are both not novel. 3) The paper mentions the use of untied parameters for premise and hypothesis, but it doesn’t compare it with tied version in the experiment section. 4) In Table 6, for CONDITIONAL tag, why the baseline models (lower total accuracies) have a 100% accuracy, but DIIN only has about a 60% accuracy?
assistant
 Pros: The paper proposes a “Densely Interactive Inference Network (DIIN)” for NLI or NLI alike tasks. Although using tensors to capture high-order interaction and performing dimension reduction over that are both not novel, the paper explores them for NLI. The paper is written clearly and is very easy to follow. The ablation experiments in Table 5 give a good level of details to help observe different components* effectiveness. Cons: 1) The differences of performances between the proposed model and the previous models are not very clear. With regard to MultiNLI, since the previous results (e.g., those in Table 2) did not use cross-sentence attention and had to represent a premise or a hypothesis as a *fixed-length* vector, is it fair to compare DIIN with them? Note that the proposed DIIN model does represent a premise or a hypothesis by variable lengths (see interaction layer in Figure 1), and tensors provide some sorts of attention between them. Can this (Table 2) really shows the advantage of the proposed models? However, when a variable-length representation is allowed (see Table 3 on SNLI), the advantage of the model is also not observed, with no improvement as a single model (compared with ESIM) and being almost same as previous models (e.g., model 18 in Table 3) in ensembling. 2) Method-wise, as discussed above, using tensors to capture high-order interaction and performing dimension reduction over that are both not novel. 3) The paper mentions the use of untied parameters for premise and hypothesis, but it doesn’t compare it with tied version in the experiment section. 4) In Table 6, for CONDITIONAL tag, why the baseline models (lower total accuracies) have a 100% accuracy, but DIIN only has about a 60% accuracy?