Beyond Bilingual: Multi-sense Word Embeddings using Multilingual Context
Word embeddings, which represent a word as a point in a vector space, have become ubiquitous to several NLP tasks. A recent line of work uses bilingual (two languages) corpora to learn a different vector for each sense of a word, by exploiting crosslingual signals to aid sense identification. We present a multi-view Bayesian non-parametric algorithm which improves multi-sense word embeddings by (a) using multilingual (i.e., more than two languages) corpora to significantly improve sense embeddings beyond what one achieves with bilingual information, and (b) uses a principled approach to learn a variable number of senses per word, in a data-driven manner. Ours is the first approach with the ability to leverage multilingual corpora efficiently for multi-sense representation learning. Experiments show that multilingual training significantly improves performance over monolingual and bilingual training, by allowing us to combine different parallel corpora to leverage multilingual context. Multilingual training yields com- parable performance to a state of the art monolingual model trained on five times more training data.
This paper discusses multi-sense embedddings and proposes learning those by using aligned text across languages. Further, the paper suggests that adding more languages helps improve word sense disambiguation (as some ambiguities can be carried across language pairs). While this idea in itself isn*t new, the authors propose a particular setup for learning multi-sense embeddings by exploiting multilingual data. Broadly this is fine, but unfortunately the paper then falls short in a number of ways. For one, the model section is unnecessarily convoluted for what is a nice idea that could be described in a far more concise fashion. Next (and more importantly), comparison with other work is lacking to such an extent that it is impossible to evaluate the merits of the proposed model in an objective fashion. This paper could be a lot stronger if the learned embeddings were evaluated in downstream tasks and evaluated against other published methods. In the current version there is too little of this, leaving us with mostly relative results between model variants and t-SNE plots that don*t really add anything to the story.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Beyond Bilingual: Multi-sense Word Embeddings using Multilingual Context
Word embeddings, which represent a word as a point in a vector space, have become ubiquitous to several NLP tasks. A recent line of work uses bilingual (two languages) corpora to learn a different vector for each sense of a word, by exploiting crosslingual signals to aid sense identification. We present a multi-view Bayesian non-parametric algorithm which improves multi-sense word embeddings by (a) using multilingual (i.e., more than two languages) corpora to significantly improve sense embeddings beyond what one achieves with bilingual information, and (b) uses a principled approach to learn a variable number of senses per word, in a data-driven manner. Ours is the first approach with the ability to leverage multilingual corpora efficiently for multi-sense representation learning. Experiments show that multilingual training significantly improves performance over monolingual and bilingual training, by allowing us to combine different parallel corpora to leverage multilingual context. Multilingual training yields com- parable performance to a state of the art monolingual model trained on five times more training data.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



This paper discusses multi-sense embedddings and proposes learning those by using aligned text across languages. Further, the paper suggests that adding more languages helps improve word sense disambiguation (as some ambiguities can be carried across language pairs). While this idea in itself isn*t new, the authors propose a particular setup for learning multi-sense embeddings by exploiting multilingual data. Broadly this is fine, but unfortunately the paper then falls short in a number of ways. For one, the model section is unnecessarily convoluted for what is a nice idea that could be described in a far more concise fashion. Next (and more importantly), comparison with other work is lacking to such an extent that it is impossible to evaluate the merits of the proposed model in an objective fashion. This paper could be a lot stronger if the learned embeddings were evaluated in downstream tasks and evaluated against other published methods. In the current version there is too little of this, leaving us with mostly relative results between model variants and t-SNE plots that don*t really add anything to the story.
assistant
This paper discusses multi-sense embedddings and proposes learning those by using aligned text across languages. Further, the paper suggests that adding more languages helps improve word sense disambiguation (as some ambiguities can be carried across language pairs). While this idea in itself isn*t new, the authors propose a particular setup for learning multi-sense embeddings by exploiting multilingual data. Broadly this is fine, but unfortunately the paper then falls short in a number of ways. For one, the model section is unnecessarily convoluted for what is a nice idea that could be described in a far more concise fashion. Next (and more importantly), comparison with other work is lacking to such an extent that it is impossible to evaluate the merits of the proposed model in an objective fashion. This paper could be a lot stronger if the learned embeddings were evaluated in downstream tasks and evaluated against other published methods. In the current version there is too little of this, leaving us with mostly relative results between model variants and t-SNE plots that don*t really add anything to the story.