title: Does GPT-4 Pass the Turing Test?
summary: We evaluated GPT-4 in a public online Turing Test. The best-performing GPT-4
  prompt passed in 41% of games, outperforming baselines set by ELIZA (27%) and
  GPT-3.5 (14%), but falling short of chance and the baseline set by human
  participants (63%). Participants' decisions were based mainly on linguistic
  style (35%) and socio-emotional traits (27%), supporting the idea that
  intelligence is not sufficient to pass the Turing Test. Participants'
  demographics, including education and familiarity with LLMs, did not predict
  detection rate, suggesting that even those who understand systems deeply and
  interact with them frequently may be susceptible to deception. Despite known
  limitations as a test of intelligence, we argue that the Turing Test continues
  to be relevant as an assessment of naturalistic communication and deception. AI
  models with the ability to masquerade as humans could have widespread societal
  consequences, and we analyse the effectiveness of different strategies and
  criteria for judging humanlikeness.
paper_authors._id: 6541cc6041676ceaa2ce8966
paper_authors.name: Cameron Jones
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-11-02T16:14:59.492Z
paper_authors._id: 642b567d5df44ff24544e584
paper_authors.avatarUrl: /avatars/f770958111b505232053cd415796dbe6.svg
paper_authors.fullname: Cameron
paper_authors.type: user
paper_authors.user: camrobjones
paper_authors._id: 6541cc6041676ceaa2ce8967
paper_authors.name: Benjamin Bergen
paper_title: Does GPT-4 Pass the Turing Test?
paper_summary: We evaluated GPT-4 in a public online Turing Test. The best-performing GPT-4
  prompt passed in 41% of games, outperforming baselines set by ELIZA (27%) and
  GPT-3.5 (14%), but falling short of chance and the baseline set by human
  participants (63%). Participants' decisions were based mainly on linguistic
  style (35%) and socio-emotional traits (27%), supporting the idea that
  intelligence is not sufficient to pass the Turing Test. Participants'
  demographics, including education and familiarity with LLMs, did not predict
  detection rate, suggesting that even those who understand systems deeply and
  interact with them frequently may be susceptible to deception. Despite known
  limitations as a test of intelligence, we argue that the Turing Test continues
  to be relevant as an assessment of naturalistic communication and deception. AI
  models with the ability to masquerade as humans could have widespread societal
  consequences, and we analyse the effectiveness of different strategies and
  criteria for judging humanlikeness.