Word translation without parallel data
State-of-the-art methods for learning cross-lingual word embeddings have relied on bilingual dictionaries or parallel corpora. Recent studies showed that the need for parallel data supervision can be alleviated with character-level information. While these methods showed encouraging results, they are not on par with their supervised counterparts and are limited to pairs of languages sharing a common alphabet. In this work, we show that we can build a bilingual dictionary between two languages without using any parallel corpora, by aligning monolingual word embedding spaces in an unsupervised way. Without using any character information, our model even outperforms existing supervised methods on cross-lingual tasks for some language pairs. Our experiments demonstrate that our method works very well also for distant language pairs, like English-Russian or English-Chinese. We finally describe experiments on the English-Esperanto low-resource language pair, on which there only exists a limited amount of parallel data, to show the potential impact of our method in fully unsupervised machine translation. Our code, embeddings and dictionaries are publicly available.
 This paper presents a new method for obtaining a bilingual dictionary, without requiring any parallel data between the source and target languages. The method consists of an adversarial approach for aligning two monolingual word embedding spaces, followed by a refinement step using frequent aligned words (according to the adversarial mapping). The approach is evaluated on single word translation, cross-lingual word similarity, and sentence translation retrieval tasks. The paper presents an interesting approach which achieves good performance. The work is presented clearly, the approach is well-motivated and related to previous studies, and a thorough evaluation is performed. My one concern is that the supervised approach that the paper compares to is limited: it is trained on a small fixed number of anchor points, while the unsupervised method uses significantly more words. I think the paper*s comparisons are valid, but the abstract and introduction make very strong claims about outperforming "state-of-the-art supervised approaches". I think either a stronger supervised baseline should be included (trained on comparable data as the unsupervised approach), or the language/claims in the paper should be softened. The same holds for statements like "... our method is a first step ...", which is very hard to justify. I also do not think it is necessary to over-sell, given the solid work in the paper. Further comments, questions and suggestions: - It might be useful to add more details of your actual approach in the Abstract, not just what it achieves. - Given you use trained word embeddings, it is not a given that the monolingual word embedding spaces would be alignable in a linear way. The actual word embedding method, therefore, has a big influence on performance (as you show). Could you comment on how crucial it would be to train monolingual embedding spaces on similar domains/data with similar co-occurrence statistics, in order for your method to be appropriate? - Would it be possible to add weights to the terms in eq. (6), or is this done implicitly? - How were the 5k source words for Procrustes supervised baseline selected? - Have you considered non-linear mappings, or jointly training the monolingual word embeddings while attempting the linear mapping between embedding spaces? - Do you think your approach would benefit from having a few parallel training points? Some minor grammatical mistakes/typos (nitpicking): - "gives a good performance" -> "gives good performance" - "Recent works", "several works", "most works", etc. -> "recent studies", "several studies", etc. - "i.e, the improvements" -> "i.e., the improvements" The paper is well-written, relevant and interesting. I therefore recommend that the paper be accepted.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Word translation without parallel data
State-of-the-art methods for learning cross-lingual word embeddings have relied on bilingual dictionaries or parallel corpora. Recent studies showed that the need for parallel data supervision can be alleviated with character-level information. While these methods showed encouraging results, they are not on par with their supervised counterparts and are limited to pairs of languages sharing a common alphabet. In this work, we show that we can build a bilingual dictionary between two languages without using any parallel corpora, by aligning monolingual word embedding spaces in an unsupervised way. Without using any character information, our model even outperforms existing supervised methods on cross-lingual tasks for some language pairs. Our experiments demonstrate that our method works very well also for distant language pairs, like English-Russian or English-Chinese. We finally describe experiments on the English-Esperanto low-resource language pair, on which there only exists a limited amount of parallel data, to show the potential impact of our method in fully unsupervised machine translation. Our code, embeddings and dictionaries are publicly available.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



 This paper presents a new method for obtaining a bilingual dictionary, without requiring any parallel data between the source and target languages. The method consists of an adversarial approach for aligning two monolingual word embedding spaces, followed by a refinement step using frequent aligned words (according to the adversarial mapping). The approach is evaluated on single word translation, cross-lingual word similarity, and sentence translation retrieval tasks. The paper presents an interesting approach which achieves good performance. The work is presented clearly, the approach is well-motivated and related to previous studies, and a thorough evaluation is performed. My one concern is that the supervised approach that the paper compares to is limited: it is trained on a small fixed number of anchor points, while the unsupervised method uses significantly more words. I think the paper*s comparisons are valid, but the abstract and introduction make very strong claims about outperforming "state-of-the-art supervised approaches". I think either a stronger supervised baseline should be included (trained on comparable data as the unsupervised approach), or the language/claims in the paper should be softened. The same holds for statements like "... our method is a first step ...", which is very hard to justify. I also do not think it is necessary to over-sell, given the solid work in the paper. Further comments, questions and suggestions: - It might be useful to add more details of your actual approach in the Abstract, not just what it achieves. - Given you use trained word embeddings, it is not a given that the monolingual word embedding spaces would be alignable in a linear way. The actual word embedding method, therefore, has a big influence on performance (as you show). Could you comment on how crucial it would be to train monolingual embedding spaces on similar domains/data with similar co-occurrence statistics, in order for your method to be appropriate? - Would it be possible to add weights to the terms in eq. (6), or is this done implicitly? - How were the 5k source words for Procrustes supervised baseline selected? - Have you considered non-linear mappings, or jointly training the monolingual word embeddings while attempting the linear mapping between embedding spaces? - Do you think your approach would benefit from having a few parallel training points? Some minor grammatical mistakes/typos (nitpicking): - "gives a good performance" -> "gives good performance" - "Recent works", "several works", "most works", etc. -> "recent studies", "several studies", etc. - "i.e, the improvements" -> "i.e., the improvements" The paper is well-written, relevant and interesting. I therefore recommend that the paper be accepted.
assistant
 This paper presents a new method for obtaining a bilingual dictionary, without requiring any parallel data between the source and target languages. The method consists of an adversarial approach for aligning two monolingual word embedding spaces, followed by a refinement step using frequent aligned words (according to the adversarial mapping). The approach is evaluated on single word translation, cross-lingual word similarity, and sentence translation retrieval tasks. The paper presents an interesting approach which achieves good performance. The work is presented clearly, the approach is well-motivated and related to previous studies, and a thorough evaluation is performed. My one concern is that the supervised approach that the paper compares to is limited: it is trained on a small fixed number of anchor points, while the unsupervised method uses significantly more words. I think the paper*s comparisons are valid, but the abstract and introduction make very strong claims about outperforming "state-of-the-art supervised approaches". I think either a stronger supervised baseline should be included (trained on comparable data as the unsupervised approach), or the language/claims in the paper should be softened. The same holds for statements like "... our method is a first step ...", which is very hard to justify. I also do not think it is necessary to over-sell, given the solid work in the paper. Further comments, questions and suggestions: - It might be useful to add more details of your actual approach in the Abstract, not just what it achieves. - Given you use trained word embeddings, it is not a given that the monolingual word embedding spaces would be alignable in a linear way. The actual word embedding method, therefore, has a big influence on performance (as you show). Could you comment on how crucial it would be to train monolingual embedding spaces on similar domains/data with similar co-occurrence statistics, in order for your method to be appropriate? - Would it be possible to add weights to the terms in eq. (6), or is this done implicitly? - How were the 5k source words for Procrustes supervised baseline selected? - Have you considered non-linear mappings, or jointly training the monolingual word embeddings while attempting the linear mapping between embedding spaces? - Do you think your approach would benefit from having a few parallel training points? Some minor grammatical mistakes/typos (nitpicking): - "gives a good performance" -> "gives good performance" - "Recent works", "several works", "most works", etc. -> "recent studies", "several studies", etc. - "i.e, the improvements" -> "i.e., the improvements" The paper is well-written, relevant and interesting. I therefore recommend that the paper be accepted.