Zero-shot Cross-Lingual Transfer for Synthetic Data Generation in
  Grammatical Error Detection
Grammatical Error Detection (GED) methods rely heavily on human annotated
error corpora. However, these annotations are unavailable in many low-resource
languages. In this paper, we investigate GED in this context. Leveraging the
zero-shot cross-lingual transfer capabilities of multilingual pre-trained
language models, we train a model using data from a diverse set of languages to
generate synthetic errors in other languages. These synthetic error corpora are
then used to train a GED model. Specifically we propose a two-stage fine-tuning
pipeline where the GED model is first fine-tuned on multilingual synthetic data
from target languages followed by fine-tuning on human-annotated GED corpora
from source languages. This approach outperforms current state-of-the-art
annotation-free GED methods. We also analyse the errors produced by our method
and other strong baselines, finding that our approach produces errors that are
more diverse and more similar to human errors.
6697e570edf0850cf9312c5a
Gaetan Lopez Latouche
claimed_verified
2024-07-18T09:06:09.263Z
64b7f95a17681d64b1b8f07a
/avatars/389c820913587a7273a05f655a1c2602.svg
Gaetan Lopez
user
GaeLop
6697e570edf0850cf9312c5b
Marc-André Carbonneau
6697e570edf0850cf9312c5c
Ben Swanson
Zero-shot Cross-Lingual Transfer for Synthetic Data Generation in
  Grammatical Error Detection
Grammatical Error Detection (GED) methods rely heavily on human annotated
error corpora. However, these annotations are unavailable in many low-resource
languages. In this paper, we investigate GED in this context. Leveraging the
zero-shot cross-lingual transfer capabilities of multilingual pre-trained
language models, we train a model using data from a diverse set of languages to
generate synthetic errors in other languages. These synthetic error corpora are
then used to train a GED model. Specifically we propose a two-stage fine-tuning
pipeline where the GED model is first fine-tuned on multilingual synthetic data
from target languages followed by fine-tuning on human-annotated GED corpora
from source languages. This approach outperforms current state-of-the-art
annotation-free GED methods. We also analyse the errors produced by our method
and other strong baselines, finding that our approach produces errors that are
more diverse and more similar to human errors.
A two-stage fine-tuning pipeline using multilingual pre-trained language models generates synthetic errors for training grammatical error detection in low-resource languages, outperforming annotation-free methods.
multilingual pre-trained language models synthetic error corpora two-stage fine-tuning grammatical error detection low-resource languages annotation-free methods