title: Zero-shot Cross-Lingual Transfer for Synthetic Data Generation in
    Grammatical Error Detection
summary: Grammatical Error Detection (GED) methods rely heavily on human annotated
  error corpora. However, these annotations are unavailable in many low-resource
  languages. In this paper, we investigate GED in this context. Leveraging the
  zero-shot cross-lingual transfer capabilities of multilingual pre-trained
  language models, we train a model using data from a diverse set of languages to
  generate synthetic errors in other languages. These synthetic error corpora are
  then used to train a GED model. Specifically we propose a two-stage fine-tuning
  pipeline where the GED model is first fine-tuned on multilingual synthetic data
  from target languages followed by fine-tuning on human-annotated GED corpora
  from source languages. This approach outperforms current state-of-the-art
  annotation-free GED methods. We also analyse the errors produced by our method
  and other strong baselines, finding that our approach produces errors that are
  more diverse and more similar to human errors.
paper_authors._id: 6697e570edf0850cf9312c5a
paper_authors.name: Gaetan Lopez Latouche
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-07-18T09:06:09.263Z
paper_authors._id: 64b7f95a17681d64b1b8f07a
paper_authors.avatarUrl: /avatars/389c820913587a7273a05f655a1c2602.svg
paper_authors.fullname: Gaetan Lopez
paper_authors.type: user
paper_authors.user: GaeLop
paper_authors._id: 6697e570edf0850cf9312c5b
paper_authors.name: Marc-André Carbonneau
paper_authors._id: 6697e570edf0850cf9312c5c
paper_authors.name: Ben Swanson
paper_title: Zero-shot Cross-Lingual Transfer for Synthetic Data Generation in
    Grammatical Error Detection
paper_summary: Grammatical Error Detection (GED) methods rely heavily on human annotated
  error corpora. However, these annotations are unavailable in many low-resource
  languages. In this paper, we investigate GED in this context. Leveraging the
  zero-shot cross-lingual transfer capabilities of multilingual pre-trained
  language models, we train a model using data from a diverse set of languages to
  generate synthetic errors in other languages. These synthetic error corpora are
  then used to train a GED model. Specifically we propose a two-stage fine-tuning
  pipeline where the GED model is first fine-tuned on multilingual synthetic data
  from target languages followed by fine-tuning on human-annotated GED corpora
  from source languages. This approach outperforms current state-of-the-art
  annotation-free GED methods. We also analyse the errors produced by our method
  and other strong baselines, finding that our approach produces errors that are
  more diverse and more similar to human errors.
paper_ai_keywords: multilingual pre-trained language models zero-shot cross-lingual transfer synthetic errors two-stage fine-tuning pipeline