XLDA: Cross-Lingual Data Augmentation for Natural Language Inference and Question Answering
While natural language processing systems often focus on a single language, multilingual transfer learning has the potential to improve performance, especially for low-resource languages. We introduce XLDA, cross-lingual data augmentation, a method that replaces a segment of the input text with its translation in another language. XLDA enhances performance of all 14 tested languages of the cross-lingual natural language inference (XNLI) benchmark. With improvements of up to 4.8, training with XLDA achieves state-of-the-art performance for Greek, Turkish, and Urdu. XLDA is in contrast to, and performs markedly better than, a more naive approach that aggregates examples in various languages in a way that each example is solely in one language. On the SQuAD question answering task, we see that XLDA provides a 1.0 performance increase on the English evaluation set. Comprehensive experiments suggest that most languages are effective as cross-lingual augmentors, that XLDA is robust to a wide range of translation quality, and that XLDA is even more effective for randomly initialized models than for pretrained models.
The paper provides an analysis of a cross-lingual data augmentation technique dubbed XLDA, which consists of replacing parts of an input text with its translation in another language. Building on the mBERT approach, the authors show that at fine-tuning time it is beneficial to augment the training set of XNLI with cross-lingual hypotheses and premises instead of in-language pairs. For each language in XNLI, they show results by augmenting with each of the 14 other languages in the dataset, and show significant improvements over per-language performance. The paper explores an interesting idea and shows that cross-lingual data augmentation works well. However, their analysis is limited to the XNLI and the Squad dataset, which do not cover a suitable range of tasks to fully conclude on the importance of XLDA for generally improving NLU tasks. It would have been interesting to show the effect of cross-lingual data augmentation for other GLUE tasks by augmenting the datasets with machine translation for instance. And also compare this model on these tasks to the monolingual back-translation approach, similar to https://arxiv.org/abs/1904.12848 . Applying XLDA with the latest open-source XLM models from the cross-lingual language model pretraining paper which obtain higher performance than the multilingual BERT would also make the results more convincing. While I share the excitement of using cross-lingual models to improve monolingual performance, I also feel like this paper lacks novelty and further evaluation to be accepted at ICLR, and would be more suited in a more NLP-focused venue.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
XLDA: Cross-Lingual Data Augmentation for Natural Language Inference and Question Answering
While natural language processing systems often focus on a single language, multilingual transfer learning has the potential to improve performance, especially for low-resource languages. We introduce XLDA, cross-lingual data augmentation, a method that replaces a segment of the input text with its translation in another language. XLDA enhances performance of all 14 tested languages of the cross-lingual natural language inference (XNLI) benchmark. With improvements of up to 4.8, training with XLDA achieves state-of-the-art performance for Greek, Turkish, and Urdu. XLDA is in contrast to, and performs markedly better than, a more naive approach that aggregates examples in various languages in a way that each example is solely in one language. On the SQuAD question answering task, we see that XLDA provides a 1.0 performance increase on the English evaluation set. Comprehensive experiments suggest that most languages are effective as cross-lingual augmentors, that XLDA is robust to a wide range of translation quality, and that XLDA is even more effective for randomly initialized models than for pretrained models.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



The paper provides an analysis of a cross-lingual data augmentation technique dubbed XLDA, which consists of replacing parts of an input text with its translation in another language. Building on the mBERT approach, the authors show that at fine-tuning time it is beneficial to augment the training set of XNLI with cross-lingual hypotheses and premises instead of in-language pairs. For each language in XNLI, they show results by augmenting with each of the 14 other languages in the dataset, and show significant improvements over per-language performance. The paper explores an interesting idea and shows that cross-lingual data augmentation works well. However, their analysis is limited to the XNLI and the Squad dataset, which do not cover a suitable range of tasks to fully conclude on the importance of XLDA for generally improving NLU tasks. It would have been interesting to show the effect of cross-lingual data augmentation for other GLUE tasks by augmenting the datasets with machine translation for instance. And also compare this model on these tasks to the monolingual back-translation approach, similar to https://arxiv.org/abs/1904.12848 . Applying XLDA with the latest open-source XLM models from the cross-lingual language model pretraining paper which obtain higher performance than the multilingual BERT would also make the results more convincing. While I share the excitement of using cross-lingual models to improve monolingual performance, I also feel like this paper lacks novelty and further evaluation to be accepted at ICLR, and would be more suited in a more NLP-focused venue.
assistant
The paper provides an analysis of a cross-lingual data augmentation technique dubbed XLDA, which consists of replacing parts of an input text with its translation in another language. Building on the mBERT approach, the authors show that at fine-tuning time it is beneficial to augment the training set of XNLI with cross-lingual hypotheses and premises instead of in-language pairs. For each language in XNLI, they show results by augmenting with each of the 14 other languages in the dataset, and show significant improvements over per-language performance. The paper explores an interesting idea and shows that cross-lingual data augmentation works well. However, their analysis is limited to the XNLI and the Squad dataset, which do not cover a suitable range of tasks to fully conclude on the importance of XLDA for generally improving NLU tasks. It would have been interesting to show the effect of cross-lingual data augmentation for other GLUE tasks by augmenting the datasets with machine translation for instance. And also compare this model on these tasks to the monolingual back-translation approach, similar to https://arxiv.org/abs/1904.12848 . Applying XLDA with the latest open-source XLM models from the cross-lingual language model pretraining paper which obtain higher performance than the multilingual BERT would also make the results more convincing. While I share the excitement of using cross-lingual models to improve monolingual performance, I also feel like this paper lacks novelty and further evaluation to be accepted at ICLR, and would be more suited in a more NLP-focused venue.