Uncovering LLM-Generated Code: A Zero-Shot Synthetic Code Detector via Code Rewriting
<|reference_start|>Uncovering LLM-Generated Code: A Zero-Shot Synthetic Code Detector via Code Rewriting: Large Language Models (LLMs) have exhibited remarkable proficiency in generating code. However, the misuse of LLM-generated (Synthetic) code has prompted concerns within both educational and industrial domains, highlighting the imperative need for the development of synthetic code detectors. Existing methods for detecting LLM-generated content are primarily tailored for general text and often struggle with code content due to the distinct grammatical structure of programming languages and massive "low-entropy" tokens. Building upon this, our work proposes a novel zero-shot synthetic code detector based on the similarity between the code and its rewritten variants. Our method relies on the intuition that the differences between the LLM-rewritten and original codes tend to be smaller when the original code is synthetic. We utilize self-supervised contrastive learning to train a code similarity model and assess our approach on two synthetic code detection benchmarks. Our results demonstrate a notable enhancement over existing synthetic content detectors designed for general texts, with an improvement of 20.5% in the APPS benchmark and 29.1% in the MBPP benchmark.<|reference_end|>
@article{ye2024uncovering,
    title={Uncovering LLM-Generated Code: A Zero-Shot Synthetic Code Detector via
  Code Rewriting},
    author={Tong Ye, Yangkai Du, Tengfei Ma, Lingfei Wu, Xuhong Zhang, Shouling
  Ji, Wenhai Wang},
    journal={arXiv preprint arXiv:2405.16133},
    year={2024},
    archivePrefix={arXiv},
    eprint={2405.16133},
    primaryClass={cs.SE cs.AI}
}