Beyond Bilingual: Multi-sense Word Embeddings using Multilingual Context
Word embeddings, which represent a word as a point in a vector space, have become ubiquitous to several NLP tasks. A recent line of work uses bilingual (two languages) corpora to learn a different vector for each sense of a word, by exploiting crosslingual signals to aid sense identification. We present a multi-view Bayesian non-parametric algorithm which improves multi-sense word embeddings by (a) using multilingual (i.e., more than two languages) corpora to significantly improve sense embeddings beyond what one achieves with bilingual information, and (b) uses a principled approach to learn a variable number of senses per word, in a data-driven manner. Ours is the first approach with the ability to leverage multilingual corpora efficiently for multi-sense representation learning. Experiments show that multilingual training significantly improves performance over monolingual and bilingual training, by allowing us to combine different parallel corpora to leverage multilingual context. Multilingual training yields com- parable performance to a state of the art monolingual model trained on five times more training data.
this work aims to address representation of multi-sense words by exploiting multilingual context. Experiments on word sense induction and word similarity in context show that the proposed solution improves over the baseline. From a computational linguistics perspective, the fact that languages less similar to English help more is intriguing. I see following problem with this work: - the paper is hard to follow and hard to see what*s new compared to the baseline model [1]. A paragraph of discussion should clearly compare and contrast with that work. - the proposed model is a slight variation of the previous work [1] thus the experimental setup should be designed in a way so that we compare which part helps improvement and how much. thus MONO has not been exposed the same training data and we can*t be sure that the proposed model is better because MONO does not observe the data or lacks the computational power. I suggest following baseline: turning multilingual data to monolingual one using the alignment, then train the baseline model[1] on this pseudo monolingual data. - the paper provides good benchmarks for intrinsic evaluation but the message could be conveyed more strongly if we see improvement in a downstream task. [1] https://arxiv.org/pdf/1502.07257v2.pdf
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Beyond Bilingual: Multi-sense Word Embeddings using Multilingual Context
Word embeddings, which represent a word as a point in a vector space, have become ubiquitous to several NLP tasks. A recent line of work uses bilingual (two languages) corpora to learn a different vector for each sense of a word, by exploiting crosslingual signals to aid sense identification. We present a multi-view Bayesian non-parametric algorithm which improves multi-sense word embeddings by (a) using multilingual (i.e., more than two languages) corpora to significantly improve sense embeddings beyond what one achieves with bilingual information, and (b) uses a principled approach to learn a variable number of senses per word, in a data-driven manner. Ours is the first approach with the ability to leverage multilingual corpora efficiently for multi-sense representation learning. Experiments show that multilingual training significantly improves performance over monolingual and bilingual training, by allowing us to combine different parallel corpora to leverage multilingual context. Multilingual training yields com- parable performance to a state of the art monolingual model trained on five times more training data.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



this work aims to address representation of multi-sense words by exploiting multilingual context. Experiments on word sense induction and word similarity in context show that the proposed solution improves over the baseline. From a computational linguistics perspective, the fact that languages less similar to English help more is intriguing. I see following problem with this work: - the paper is hard to follow and hard to see what*s new compared to the baseline model [1]. A paragraph of discussion should clearly compare and contrast with that work. - the proposed model is a slight variation of the previous work [1] thus the experimental setup should be designed in a way so that we compare which part helps improvement and how much. thus MONO has not been exposed the same training data and we can*t be sure that the proposed model is better because MONO does not observe the data or lacks the computational power. I suggest following baseline: turning multilingual data to monolingual one using the alignment, then train the baseline model[1] on this pseudo monolingual data. - the paper provides good benchmarks for intrinsic evaluation but the message could be conveyed more strongly if we see improvement in a downstream task. [1] https://arxiv.org/pdf/1502.07257v2.pdf
assistant
this work aims to address representation of multi-sense words by exploiting multilingual context. Experiments on word sense induction and word similarity in context show that the proposed solution improves over the baseline. From a computational linguistics perspective, the fact that languages less similar to English help more is intriguing. I see following problem with this work: - the paper is hard to follow and hard to see what*s new compared to the baseline model [1]. A paragraph of discussion should clearly compare and contrast with that work. - the proposed model is a slight variation of the previous work [1] thus the experimental setup should be designed in a way so that we compare which part helps improvement and how much. thus MONO has not been exposed the same training data and we can*t be sure that the proposed model is better because MONO does not observe the data or lacks the computational power. I suggest following baseline: turning multilingual data to monolingual one using the alignment, then train the baseline model[1] on this pseudo monolingual data. - the paper provides good benchmarks for intrinsic evaluation but the message could be conveyed more strongly if we see improvement in a downstream task. [1] https://arxiv.org/pdf/1502.07257v2.pdf