Unsupervised Latent Tree Induction with Deep Inside-Outside Recursive Auto-Encoders
Syntax is a powerful abstraction for language understanding. Many downstream tasks require segmenting input text into meaningful constituent chunks (e.g., noun phrases or entities); more generally, models for learning semantic representations of text benefit from integrating syntax in the form of parse trees (e.g., tree-LSTMs). Supervised parsers have traditionally been used to obtain these trees, but lately interest has increased in unsupervised methods that induce syntactic representations directly from unlabeled text. To this end, we propose the deep inside-outside recursive autoencoder (DIORA), a fully-unsupervised method for discovering syntax that simultaneously learns representations for constituents within the induced tree. Unlike many prior approaches, DIORA does not rely on supervision from auxiliary downstream tasks and is thus not constrained to particular domains. Furthermore, competing approaches do not learn explicit phrase representations along with tree structures, which limits their applicability to phrase-based tasks. Extensive experiments on unsupervised parsing, segmentation, and phrase clustering demonstrate the efficacy of our method. DIORA achieves the state of the art in unsupervised parsing (46.9 F1) on the benchmark WSJ dataset.
 This paper proposes a model for unsupervised dependency parsing (latent tree induction) that is based on a combination of the inside-outside algorithm with neural modeling (recursive auto-encoders). The paper is clearly written and the model seems interesting and, as far as I can tell, also novel. Yet, the paper suffers from a major limitation that deems its rejection. Unfortunately, the authors seem to be totally unaware of previous work on unsupervised parsing. It may sound surprising but people have worked on this problem even before 2014 and there are some strong results that do not involve deep learning. I am sorry for the sarcasm, but it was really frustrating to see not only the lack of citation, but also the statement (already in the abstract and throughout the paper) that an F-score of 46.9 set a SOTA for unsupervised parsing with WSJ PennTreebank. Particularly, the authors ignore works of Cohen and Smith, Spitkovsy et al., Seginer and many others. A quick look at Spitkosky et al., 2013 (EMNLP) would reveal that the reported result is not SOTA (although Spitkovsky et al report results for sentences no longer than 40 words, but given the numbers they report for several models - an F1 score of 54 and more - the full WSJ number is likely to be higher than 46.9). But all the papers cited at Spitkovsky et al. 2013 are even not cited here. I would also like to refer the authors to work by Cohen and Collins (2012-2013) on latent variable PCFG, which presents a provably consistent parameter estimation for the problem. The presented techniques may also be an interesting point of comparison to this work and phrase representation may also be extracted from that algorithm.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Unsupervised Latent Tree Induction with Deep Inside-Outside Recursive Auto-Encoders
Syntax is a powerful abstraction for language understanding. Many downstream tasks require segmenting input text into meaningful constituent chunks (e.g., noun phrases or entities); more generally, models for learning semantic representations of text benefit from integrating syntax in the form of parse trees (e.g., tree-LSTMs). Supervised parsers have traditionally been used to obtain these trees, but lately interest has increased in unsupervised methods that induce syntactic representations directly from unlabeled text. To this end, we propose the deep inside-outside recursive autoencoder (DIORA), a fully-unsupervised method for discovering syntax that simultaneously learns representations for constituents within the induced tree. Unlike many prior approaches, DIORA does not rely on supervision from auxiliary downstream tasks and is thus not constrained to particular domains. Furthermore, competing approaches do not learn explicit phrase representations along with tree structures, which limits their applicability to phrase-based tasks. Extensive experiments on unsupervised parsing, segmentation, and phrase clustering demonstrate the efficacy of our method. DIORA achieves the state of the art in unsupervised parsing (46.9 F1) on the benchmark WSJ dataset.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



 This paper proposes a model for unsupervised dependency parsing (latent tree induction) that is based on a combination of the inside-outside algorithm with neural modeling (recursive auto-encoders). The paper is clearly written and the model seems interesting and, as far as I can tell, also novel. Yet, the paper suffers from a major limitation that deems its rejection. Unfortunately, the authors seem to be totally unaware of previous work on unsupervised parsing. It may sound surprising but people have worked on this problem even before 2014 and there are some strong results that do not involve deep learning. I am sorry for the sarcasm, but it was really frustrating to see not only the lack of citation, but also the statement (already in the abstract and throughout the paper) that an F-score of 46.9 set a SOTA for unsupervised parsing with WSJ PennTreebank. Particularly, the authors ignore works of Cohen and Smith, Spitkovsy et al., Seginer and many others. A quick look at Spitkosky et al., 2013 (EMNLP) would reveal that the reported result is not SOTA (although Spitkovsky et al report results for sentences no longer than 40 words, but given the numbers they report for several models - an F1 score of 54 and more - the full WSJ number is likely to be higher than 46.9). But all the papers cited at Spitkovsky et al. 2013 are even not cited here. I would also like to refer the authors to work by Cohen and Collins (2012-2013) on latent variable PCFG, which presents a provably consistent parameter estimation for the problem. The presented techniques may also be an interesting point of comparison to this work and phrase representation may also be extracted from that algorithm.
assistant
 This paper proposes a model for unsupervised dependency parsing (latent tree induction) that is based on a combination of the inside-outside algorithm with neural modeling (recursive auto-encoders). The paper is clearly written and the model seems interesting and, as far as I can tell, also novel. Yet, the paper suffers from a major limitation that deems its rejection. Unfortunately, the authors seem to be totally unaware of previous work on unsupervised parsing. It may sound surprising but people have worked on this problem even before 2014 and there are some strong results that do not involve deep learning. I am sorry for the sarcasm, but it was really frustrating to see not only the lack of citation, but also the statement (already in the abstract and throughout the paper) that an F-score of 46.9 set a SOTA for unsupervised parsing with WSJ PennTreebank. Particularly, the authors ignore works of Cohen and Smith, Spitkovsy et al., Seginer and many others. A quick look at Spitkosky et al., 2013 (EMNLP) would reveal that the reported result is not SOTA (although Spitkovsky et al report results for sentences no longer than 40 words, but given the numbers they report for several models - an F1 score of 54 and more - the full WSJ number is likely to be higher than 46.9). But all the papers cited at Spitkovsky et al. 2013 are even not cited here. I would also like to refer the authors to work by Cohen and Collins (2012-2013) on latent variable PCFG, which presents a provably consistent parameter estimation for the problem. The presented techniques may also be an interesting point of comparison to this work and phrase representation may also be extracted from that algorithm.