title: DNA-GPT: Divergent N-Gram Analysis for Training-Free Detection of
    GPT-Generated Text
summary: Large language models (LLMs) have notably enhanced the fluency and diversity
  of machine-generated text. However, this progress also presents a significant
  challenge in detecting the origin of a given text, and current research on
  detection methods lags behind the rapid evolution of LLMs. Conventional
  training-based methods have limitations in flexibility, particularly when
  adapting to new domains, and they often lack explanatory power. To address this
  gap, we propose a novel training-free detection strategy called Divergent
  N-Gram Analysis (DNA-GPT). Given a text, we first truncate it in the middle and
  then use only the preceding portion as input to the LLMs to regenerate the new
  remaining parts. By analyzing the differences between the original and new
  remaining parts through N-gram analysis in black-box or probability divergence
  in white-box, we can clearly illustrate significant discrepancies between
  machine-generated and human-written text. We conducted extensive experiments on
  the most advanced LLMs from OpenAI, including text-davinci-003, GPT-3.5-turbo,
  and GPT-4, as well as open-source models such as GPT-NeoX-20B and LLaMa-13B.
  Results show that our zero-shot approach exhibits state-of-the-art performance
  in distinguishing between human and GPT-generated text on four English and one
  German dataset, outperforming OpenAI's own classifier, which is trained on
  millions of text. Additionally, our methods provide reasonable explanations and
  evidence to support our claim, which is a unique feature of explainable
  detection. Our method is also robust under the revised text attack and can
  additionally solve model sourcing. Codes are available at
  https://github.com/Xianjun-Yang/DNA-GPT.
paper_authors._id: 64755c0f7d131daf6340de73
paper_authors.name: Xianjun Yang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-30T18:00:11.655Z
paper_authors._id: 604137e5c15e823a685f2af2
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/604137e5c15e823a685f2af2/rrxZUyrbLpKqCmG6xtP8v.jpeg
paper_authors.fullname: Xianjun Yang
paper_authors.type: user
paper_authors.user: Xianjun
paper_authors._id: 64755c0f7d131daf6340de74
paper_authors.name: Wei Cheng
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-30T14:15:24.520Z
paper_authors._id: 6475f47d4766357252e7bc2a
paper_authors.avatarUrl: /avatars/2fa3772318619258b25dd83533e74cea.svg
paper_authors.fullname: Wei Cheng
paper_authors.type: user
paper_authors.user: chengw07
paper_authors._id: 64755c0f7d131daf6340de75
paper_authors.name: Linda Petzold
paper_authors._id: 64755c0f7d131daf6340de76
paper_authors.name: William Yang Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-31T07:24:35.872Z
paper_authors._id: 6476607efb22e3b77f3f1452
paper_authors.avatarUrl: /avatars/22f40c81e6b6e09bdb66a5afd978f98b.svg
paper_authors.fullname: William Yang Wang
paper_authors.type: user
paper_authors.user: wangwilliamyang
paper_authors._id: 64755c0f7d131daf6340de77
paper_authors.name: Haifeng Chen
paper_title: DNA-GPT: Divergent N-Gram Analysis for Training-Free Detection of
    GPT-Generated Text
paper_summary: Large language models (LLMs) have notably enhanced the fluency and diversity
  of machine-generated text. However, this progress also presents a significant
  challenge in detecting the origin of a given text, and current research on
  detection methods lags behind the rapid evolution of LLMs. Conventional
  training-based methods have limitations in flexibility, particularly when
  adapting to new domains, and they often lack explanatory power. To address this
  gap, we propose a novel training-free detection strategy called Divergent
  N-Gram Analysis (DNA-GPT). Given a text, we first truncate it in the middle and
  then use only the preceding portion as input to the LLMs to regenerate the new
  remaining parts. By analyzing the differences between the original and new
  remaining parts through N-gram analysis in black-box or probability divergence
  in white-box, we can clearly illustrate significant discrepancies between
  machine-generated and human-written text. We conducted extensive experiments on
  the most advanced LLMs from OpenAI, including text-davinci-003, GPT-3.5-turbo,
  and GPT-4, as well as open-source models such as GPT-NeoX-20B and LLaMa-13B.
  Results show that our zero-shot approach exhibits state-of-the-art performance
  in distinguishing between human and GPT-generated text on four English and one
  German dataset, outperforming OpenAI's own classifier, which is trained on
  millions of text. Additionally, our methods provide reasonable explanations and
  evidence to support our claim, which is a unique feature of explainable
  detection. Our method is also robust under the revised text attack and can
  additionally solve model sourcing. Codes are available at
  https://github.com/Xianjun-Yang/DNA-GPT.
paper_ai_summary: A novel training-free method, Divergent N-Gram Analysis (DNA-GPT), distinguishes machine-generated text from human-written text by analyzing N-gram divergence and outperforms existing methods in detecting text origins.
paper_ai_keywords: Divergent N-Gram Analysis DNA-GPT N-gram analysis probability divergence large language models LLMs text-davinci-003 GPT-3.5-turbo GPT-4 GPT-NeoX-20B LLaMa-13B zero-shot approach model sourcing explainable detection