CLEME20: Towards More Interpretable Evaluation by Disentangling Edits for Grammatical Error Correction
<|reference_start|>CLEME20: Towards More Interpretable Evaluation by Disentangling Edits for Grammatical Error Correction: The paper focuses on improving the interpretability of Grammatical Error Correction (GEC) metrics, which receives little attention in previous studies. To bridge the gap, we propose CLEME2.0, a reference-based evaluation strategy that can describe four elementary dimensions of GEC systems, namely hit-correction, error-correction, under-correction, and over-correction. They collectively contribute to revealing the critical characteristics and locating drawbacks of GEC systems. Evaluating systems by Combining these dimensions leads to high human consistency over other reference-based and reference-less metrics. Extensive experiments on 2 human judgement datasets and 6 reference datasets demonstrate the effectiveness and robustness of our method. All the codes will be released after the peer review.<|reference_end|>
@article{ye2024cleme2.0:,
    title={CLEME2.0: Towards More Interpretable Evaluation by Disentangling Edits
  for Grammatical Error Correction},
    author={Jingheng Ye, Zishan Xu, Yinghui Li, Xuxin Cheng, Linlin Song, Qingyu
  Zhou, Hai-Tao Zheng, Ying Shen, Xin Su},
    journal={arXiv preprint arXiv:2407.00934},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.00934},
    primaryClass={cs.CL}
}