Bot or Human? Detecting ChatGPT Imposters with A Single Question
Large language models like ChatGPT have recently demonstrated impressive
capabilities in natural language understanding and generation, enabling various
applications including translation, essay writing, and chit-chatting. However,
there is a concern that they can be misused for malicious purposes, such as
fraud or denial-of-service attacks. Therefore, it is crucial to develop methods
for detecting whether the party involved in a conversation is a bot or a human.
In this paper, we propose a framework named FLAIR, Finding Large language model
Authenticity via a single Inquiry and Response, to detect conversational bots
in an online manner. Specifically, we target a single question scenario that
can effectively differentiate human users from bots. The questions are divided
into two categories: those that are easy for humans but difficult for bots
(e.g., counting, substitution, positioning, noise filtering, and ASCII art),
and those that are easy for bots but difficult for humans (e.g., memorization
and computation). Our approach shows different strengths of these questions in
their effectiveness, providing a new way for online service providers to
protect themselves against nefarious activities and ensure that they are
serving real users. We open-sourced our dataset on
https://github.com/hongwang600/FLAIR and welcome contributions from the
community to enrich such detection datasets.
645da115ce72244df7b4749e
Hong Wang
645da115ce72244df7b4749f
Xuan Luo
claimed_verified
2025-04-02T08:59:03.218Z
659d98feec9f8f337f145ec4
/avatars/7de3bea0f203326e78916882655b5d55.svg
xuan luo
user
xuan-luo
645da115ce72244df7b474a0
Weizhi Wang
claimed_verified
2023-05-16T09:55:52.832Z
63d34004b734eaa4d4faeccf
https://cdn-avatars.huggingface.co/v1/production/uploads/63d34004b734eaa4d4faeccf/aH7F-xn4PMQjHHcvL-0vs.jpeg
Weizhi Wang
user
weizhiwang
645da115ce72244df7b474a1
Xifeng Yan
Bot or Human? Detecting ChatGPT Imposters with A Single Question
Large language models like ChatGPT have recently demonstrated impressive
capabilities in natural language understanding and generation, enabling various
applications including translation, essay writing, and chit-chatting. However,
there is a concern that they can be misused for malicious purposes, such as
fraud or denial-of-service attacks. Therefore, it is crucial to develop methods
for detecting whether the party involved in a conversation is a bot or a human.
In this paper, we propose a framework named FLAIR, Finding Large language model
Authenticity via a single Inquiry and Response, to detect conversational bots
in an online manner. Specifically, we target a single question scenario that
can effectively differentiate human users from bots. The questions are divided
into two categories: those that are easy for humans but difficult for bots
(e.g., counting, substitution, positioning, noise filtering, and ASCII art),
and those that are easy for bots but difficult for humans (e.g., memorization
and computation). Our approach shows different strengths of these questions in
their effectiveness, providing a new way for online service providers to
protect themselves against nefarious activities and ensure that they are
serving real users. We open-sourced our dataset on
https://github.com/hongwang600/FLAIR and welcome contributions from the
community to enrich such detection datasets.
FLAIR framework detects conversational bots online using targeted questions to differentiate between human and artificial responses.
large language models conversational bots authentication differentiation counting substitution positioning noise filtering ASCII art memorization computation