Who Wrote This? The Key to Zero-Shot LLM-Generated Text Detection Is GECScore
<|reference_start|>Who Wrote This? The Key to Zero-Shot LLM-Generated Text Detection Is GECScore: The efficacy of an large language model (LLM) generated text detector depends substantially on the availability of sizable training data. White-box zero-shot detectors, which require no such data, are nonetheless limited by the accessibility of the source model of the LLM-generated text. In this paper, we propose an simple but effective black-box zero-shot detection approach, predicated on the observation that human-written texts typically contain more grammatical errors than LLM-generated texts. This approach entails computing the Grammar Error Correction Score (GECScore) for the given text to distinguish between human-written and LLM-generated text. Extensive experimental results show that our method outperforms current state-of-the-art (SOTA) zero-shot and supervised methods, achieving an average AUROC of 98.7% and showing strong robustness against paraphrase and adversarial perturbation attacks.<|reference_end|>
@article{wu2024who,
    title={Who Wrote This? The Key to Zero-Shot LLM-Generated Text Detection Is
  GECScore},
    author={Junchao Wu, Runzhe Zhan, Derek F. Wong, Shu Yang, Xuebo Liu, Lidia S.
  Chao, Min Zhang},
    journal={arXiv preprint arXiv:2405.04286},
    year={2024},
    archivePrefix={arXiv},
    eprint={2405.04286},
    primaryClass={cs.CL}
}