GPT-35 for Grammatical Error Correction
<|reference_start|>GPT-35 for Grammatical Error Correction: This paper investigates the application of GPT-3.5 for Grammatical Error Correction (GEC) in multiple languages in several settings: zero-shot GEC, fine-tuning for GEC, and using GPT-3.5 to re-rank correction hypotheses generated by other GEC models. In the zero-shot setting, we conduct automatic evaluations of the corrections proposed by GPT-3.5 using several methods: estimating grammaticality with language models (LMs), the Scribendi test, and comparing the semantic embeddings of sentences. GPT-3.5 has a known tendency to over-correct erroneous sentences and propose alternative corrections. For several languages, such as Czech, German, Russian, Spanish, and Ukrainian, GPT-3.5 substantially alters the source sentences, including their semantics, which presents significant challenges for evaluation with reference-based metrics. For English, GPT-3.5 demonstrates high recall, generates fluent corrections, and generally preserves sentence semantics. However, human evaluation for both English and Russian reveals that, despite its strong error-detection capabilities, GPT-3.5 struggles with several error types, including punctuation mistakes, tense errors, syntactic dependencies between words, and lexical compatibility at the sentence level.<|reference_end|>
@article{katinskaia2024gpt-3.5,
    title={GPT-3.5 for Grammatical Error Correction},
    author={Anisia Katinskaia and Roman Yangarber},
    journal={arXiv preprint arXiv:2405.08469},
    year={2024},
    archivePrefix={arXiv},
    eprint={2405.08469},
    primaryClass={cs.CL cs.AI}
}