Learning Corresponded Rationales for Text Matching
The ability to predict matches between two sources of text has a number of applications including natural language inference (NLI) and question answering (QA). While flexible neural models have become effective tools in solving these tasks, they are rarely transparent in terms of the mechanism that mediates the prediction. In this paper, we propose a self-explaining architecture where the model is forced to highlight, in a dependent manner, how spans of one side of the input match corresponding segments of the other side in order to arrive at the overall decision. The text spans are regularized to be coherent and concise, and their correspondence is captured explicitly. The text spans -- rationales -- are learned entirely as latent mechanisms, guided only by the distal supervision from the end-to-end task. We evaluate our model on both NLI and QA using three publicly available datasets. Experimental results demonstrate quantitatively and qualitatively that our method delivers interpretable justification of the prediction without sacrificing state-of-the-art performance. Our code and data split will be publicly available.
 This paper is about learning paired rationales that include the corresponding relevant spans of the (question, passage) or (premise, hypothesis). Experimental results show the same or better accuracies using just the fraction of the input selected as when the whole input is used. While there has been prior work on learning rationales, this is the first I have seen that included this fine-grained pairing. The paper also learns these rationales without explicitly labeled rationales but rather with only the distant supervision of the overall question answering or natural language inference task. This paper could be made stronger by including an experimental evaluation of accuracy in an adversarial setting. The model developed here might be well-suited for adversarial SquAD examples in which an extra sentence has been added. It would be interesting to see these results. This paper does include a somewhat similar adversarial evaluation (Section 4.3) but adds extra information to NLI examples. Since for NLI, unlike QA, the extra sentence can change the correct label (can flip from entailment to contradiction), accuracy was not able to be evaluated. Experimentally, it would be good to compare against some prior work that doesn*t include the pairing. Perhaps an interpretability model based on the passage only without fine-grained pairing with the question? My apologies if this corresponds to "Independent", I was somewhat confused by descriptions of the baseline. The descriptions of the baselines was the least clear part of this paper. It would be helpful to improve the clarity of Section 4.1 (perhaps adding a figure). Optional suggestion: consider breaking up the experiment section into two subsections: one for the cases in which the question rationales are provided (results in Table 1), and one for the cases in which the question-side rationales are learned as well (Table 2). By putting all descriptions together, the paper explains two different settings and then needs to discuss which baselines are applicable to each setting and dataset and why.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Learning Corresponded Rationales for Text Matching
The ability to predict matches between two sources of text has a number of applications including natural language inference (NLI) and question answering (QA). While flexible neural models have become effective tools in solving these tasks, they are rarely transparent in terms of the mechanism that mediates the prediction. In this paper, we propose a self-explaining architecture where the model is forced to highlight, in a dependent manner, how spans of one side of the input match corresponding segments of the other side in order to arrive at the overall decision. The text spans are regularized to be coherent and concise, and their correspondence is captured explicitly. The text spans -- rationales -- are learned entirely as latent mechanisms, guided only by the distal supervision from the end-to-end task. We evaluate our model on both NLI and QA using three publicly available datasets. Experimental results demonstrate quantitatively and qualitatively that our method delivers interpretable justification of the prediction without sacrificing state-of-the-art performance. Our code and data split will be publicly available.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



 This paper is about learning paired rationales that include the corresponding relevant spans of the (question, passage) or (premise, hypothesis). Experimental results show the same or better accuracies using just the fraction of the input selected as when the whole input is used. While there has been prior work on learning rationales, this is the first I have seen that included this fine-grained pairing. The paper also learns these rationales without explicitly labeled rationales but rather with only the distant supervision of the overall question answering or natural language inference task. This paper could be made stronger by including an experimental evaluation of accuracy in an adversarial setting. The model developed here might be well-suited for adversarial SquAD examples in which an extra sentence has been added. It would be interesting to see these results. This paper does include a somewhat similar adversarial evaluation (Section 4.3) but adds extra information to NLI examples. Since for NLI, unlike QA, the extra sentence can change the correct label (can flip from entailment to contradiction), accuracy was not able to be evaluated. Experimentally, it would be good to compare against some prior work that doesn*t include the pairing. Perhaps an interpretability model based on the passage only without fine-grained pairing with the question? My apologies if this corresponds to "Independent", I was somewhat confused by descriptions of the baseline. The descriptions of the baselines was the least clear part of this paper. It would be helpful to improve the clarity of Section 4.1 (perhaps adding a figure). Optional suggestion: consider breaking up the experiment section into two subsections: one for the cases in which the question rationales are provided (results in Table 1), and one for the cases in which the question-side rationales are learned as well (Table 2). By putting all descriptions together, the paper explains two different settings and then needs to discuss which baselines are applicable to each setting and dataset and why.
assistant
 This paper is about learning paired rationales that include the corresponding relevant spans of the (question, passage) or (premise, hypothesis). Experimental results show the same or better accuracies using just the fraction of the input selected as when the whole input is used. While there has been prior work on learning rationales, this is the first I have seen that included this fine-grained pairing. The paper also learns these rationales without explicitly labeled rationales but rather with only the distant supervision of the overall question answering or natural language inference task. This paper could be made stronger by including an experimental evaluation of accuracy in an adversarial setting. The model developed here might be well-suited for adversarial SquAD examples in which an extra sentence has been added. It would be interesting to see these results. This paper does include a somewhat similar adversarial evaluation (Section 4.3) but adds extra information to NLI examples. Since for NLI, unlike QA, the extra sentence can change the correct label (can flip from entailment to contradiction), accuracy was not able to be evaluated. Experimentally, it would be good to compare against some prior work that doesn*t include the pairing. Perhaps an interpretability model based on the passage only without fine-grained pairing with the question? My apologies if this corresponds to "Independent", I was somewhat confused by descriptions of the baseline. The descriptions of the baselines was the least clear part of this paper. It would be helpful to improve the clarity of Section 4.1 (perhaps adding a figure). Optional suggestion: consider breaking up the experiment section into two subsections: one for the cases in which the question rationales are provided (results in Table 1), and one for the cases in which the question-side rationales are learned as well (Table 2). By putting all descriptions together, the paper explains two different settings and then needs to discuss which baselines are applicable to each setting and dataset and why.