STRUCTURED ALIGNMENT NETWORKS
Many tasks in natural language processing involve comparing two sentences to compute some notion of relevance, entailment, or similarity. Typically this comparison is done either at the word level or at the sentence level, with no attempt to leverage the inherent structure of the sentence. When sentence structure is used for comparison, it is obtained during a non-differentiable pre-processing step, leading to propagation of errors. We introduce a model of structured alignments between sentences, showing how to compare two sentences by matching their latent structures. Using a structured attention mechanism, our model matches possible spans in the first sentence to possible spans in the second sentence, simultaneously discovering the tree structure of each sentence and performing a comparison, in a model that is fully differentiable and is trained only on the comparison objective. We evaluate this model on two sentence comparison tasks: the Stanford natural language inference dataset and the TREC-QA dataset. We find that comparing spans results in superior performance to comparing words individually, and that the learned trees are consistent with actual linguistic structures.
 Summary: This paper introduces a structured attention mechanisms to compute alignment scores among all possible spans in two given sentences. The span representations are weighted by the spans marginal scores given by the inside-outside algorithm. Experiments on TREC-QA and SNLI show modest improvement over the word-based structured attention baseline (Parikh et al., 2016). Strengths: The idea of using latent syntactic structure, and computing cross-sentence alignment over spans is very interesting. Weaknesses: The paper is 8.5 pages long. The method did not out-perform other very related structured attention methods (86.8, Kim et al., 2017, 86.9, Liu and Lapata, 2017) Aside from the time complexity from the inside-outside algorithm (as mentioned by the authors in conclusion), the comparison among all pairs of spans is O(n^4), which is more expensive. Am I missing something about the algorithm? It would be nice to show, quantitatively, the agreement between the latent trees and gold/supervised syntax. The paper claimed “the model is able to recover tree structures that very closely mimic syntax”, but it’s hard to draw this conclusion from the two examples in Figure 2.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
STRUCTURED ALIGNMENT NETWORKS
Many tasks in natural language processing involve comparing two sentences to compute some notion of relevance, entailment, or similarity. Typically this comparison is done either at the word level or at the sentence level, with no attempt to leverage the inherent structure of the sentence. When sentence structure is used for comparison, it is obtained during a non-differentiable pre-processing step, leading to propagation of errors. We introduce a model of structured alignments between sentences, showing how to compare two sentences by matching their latent structures. Using a structured attention mechanism, our model matches possible spans in the first sentence to possible spans in the second sentence, simultaneously discovering the tree structure of each sentence and performing a comparison, in a model that is fully differentiable and is trained only on the comparison objective. We evaluate this model on two sentence comparison tasks: the Stanford natural language inference dataset and the TREC-QA dataset. We find that comparing spans results in superior performance to comparing words individually, and that the learned trees are consistent with actual linguistic structures.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



 Summary: This paper introduces a structured attention mechanisms to compute alignment scores among all possible spans in two given sentences. The span representations are weighted by the spans marginal scores given by the inside-outside algorithm. Experiments on TREC-QA and SNLI show modest improvement over the word-based structured attention baseline (Parikh et al., 2016). Strengths: The idea of using latent syntactic structure, and computing cross-sentence alignment over spans is very interesting. Weaknesses: The paper is 8.5 pages long. The method did not out-perform other very related structured attention methods (86.8, Kim et al., 2017, 86.9, Liu and Lapata, 2017) Aside from the time complexity from the inside-outside algorithm (as mentioned by the authors in conclusion), the comparison among all pairs of spans is O(n^4), which is more expensive. Am I missing something about the algorithm? It would be nice to show, quantitatively, the agreement between the latent trees and gold/supervised syntax. The paper claimed “the model is able to recover tree structures that very closely mimic syntax”, but it’s hard to draw this conclusion from the two examples in Figure 2.
assistant
 Summary: This paper introduces a structured attention mechanisms to compute alignment scores among all possible spans in two given sentences. The span representations are weighted by the spans marginal scores given by the inside-outside algorithm. Experiments on TREC-QA and SNLI show modest improvement over the word-based structured attention baseline (Parikh et al., 2016). Strengths: The idea of using latent syntactic structure, and computing cross-sentence alignment over spans is very interesting. Weaknesses: The paper is 8.5 pages long. The method did not out-perform other very related structured attention methods (86.8, Kim et al., 2017, 86.9, Liu and Lapata, 2017) Aside from the time complexity from the inside-outside algorithm (as mentioned by the authors in conclusion), the comparison among all pairs of spans is O(n^4), which is more expensive. Am I missing something about the algorithm? It would be nice to show, quantitatively, the agreement between the latent trees and gold/supervised syntax. The paper claimed “the model is able to recover tree structures that very closely mimic syntax”, but it’s hard to draw this conclusion from the two examples in Figure 2.