title: Human or Not? A Gamified Approach to the Turing Test
summary: We present "Human or Not?", an online game inspired by the Turing test, that
  measures the capability of AI chatbots to mimic humans in dialog, and of humans
  to tell bots from other humans. Over the course of a month, the game was played
  by over 1.5 million users who engaged in anonymous two-minute chat sessions
  with either another human or an AI language model which was prompted to behave
  like humans. The task of the players was to correctly guess whether they spoke
  to a person or to an AI. This largest scale Turing-style test conducted to date
  revealed some interesting facts. For example, overall users guessed the
  identity of their partners correctly in only 68% of the games. In the subset of
  the games in which users faced an AI bot, users had even lower correct guess
  rates of 60% (that is, not much higher than chance). This white paper details
  the development, deployment, and results of this unique experiment. While this
  experiment calls for many extensions and refinements, these findings already
  begin to shed light on the inevitable near future which will commingle humans
  and AI.
paper_authors._id: 647805286e6c7ac608c56626
paper_authors.name: Daniel Jannai
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2023-06-01T07:41:31.797Z
paper_authors._id: 62d03afd485856cd710c3377
paper_authors.avatarUrl: /avatars/09613eba65ddc96301b289e860f9b3f9.svg
paper_authors.fullname: Daniel Jannai-Epstein
paper_authors.type: user
paper_authors.user: danieljannai
paper_authors._id: 647805286e6c7ac608c56627
paper_authors.name: Amos Meron
paper_authors._id: 647805286e6c7ac608c56628
paper_authors.name: Barak Lenz
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2023-06-01T05:50:09.546Z
paper_authors._id: 6339882037faf14d3a955b4a
paper_authors.avatarUrl: /avatars/5c5105d7aabe0a29bba82a96771f1767.svg
paper_authors.fullname: Lenz
paper_authors.type: user
paper_authors.user: barakl
paper_authors._id: 647805286e6c7ac608c56629
paper_authors.name: Yoav Levine
paper_authors._id: 647805286e6c7ac608c5662a
paper_authors.name: Yoav Shoham
paper_title: Human or Not? A Gamified Approach to the Turing Test
paper_summary: We present "Human or Not?", an online game inspired by the Turing test, that
  measures the capability of AI chatbots to mimic humans in dialog, and of humans
  to tell bots from other humans. Over the course of a month, the game was played
  by over 1.5 million users who engaged in anonymous two-minute chat sessions
  with either another human or an AI language model which was prompted to behave
  like humans. The task of the players was to correctly guess whether they spoke
  to a person or to an AI. This largest scale Turing-style test conducted to date
  revealed some interesting facts. For example, overall users guessed the
  identity of their partners correctly in only 68% of the games. In the subset of
  the games in which users faced an AI bot, users had even lower correct guess
  rates of 60% (that is, not much higher than chance). This white paper details
  the development, deployment, and results of this unique experiment. While this
  experiment calls for many extensions and refinements, these findings already
  begin to shed light on the inevitable near future which will commingle humans
  and AI.