LLM-Detector: Improving AI-Generated Chinese Text Detection with Open-Source LLM Instruction Tuning
<|reference_start|>LLM-Detector: Improving AI-Generated Chinese Text Detection with Open-Source LLM Instruction Tuning: ChatGPT and other general large language models (LLMs) have achieved remarkable success, but they have also raised concerns about the misuse of AI-generated texts. Existing AI-generated text detection models, such as based on BERT and RoBERTa, are prone to in-domain over-fitting, leading to poor out-of-domain (OOD) detection performance. In this paper, we first collected Chinese text responses generated by human experts and 9 types of LLMs, for which to multiple domains questions, and further created a dataset that mixed human-written sentences and sentences polished by LLMs. We then proposed LLM-Detector, a novel method for both document-level and sentence-level text detection through Instruction Tuning of LLMs. Our method leverages the wealth of knowledge LLMs acquire during pre-training, enabling them to detect the text they generate. Instruction tuning aligns the model's responses with the user's expected text detection tasks. Experimental results show that previous methods struggle with sentence-level AI-generated text detection and OOD detection. In contrast, our proposed method not only significantly outperforms baseline methods in both sentence-level and document-level text detection but also demonstrates strong generalization capabilities. Furthermore, since LLM-Detector is trained based on open-source LLMs, it is easy to customize for deployment.<|reference_end|>
@article{wang2024llm-detector:,
    title={LLM-Detector: Improving AI-Generated Chinese Text Detection with
  Open-Source LLM Instruction Tuning},
    author={Rongsheng Wang and Haoming Chen and Ruizhe Zhou and Han Ma and Yaofei
  Duan and Yanlan Kang and Songhua Yang and Baoyu Fan and Tao Tan},
    journal={arXiv preprint arXiv:2402.01158},
    year={2024},
    archivePrefix={arXiv},
    eprint={2402.01158},
    primaryClass={cs.CL}
}