Does GPT-4 Pass the Turing Test?
We evaluated GPT-4 in a public online Turing Test. The best-performing GPT-4
prompt passed in 41% of games, outperforming baselines set by ELIZA (27%) and
GPT-3.5 (14%), but falling short of chance and the baseline set by human
participants (63%). Participants' decisions were based mainly on linguistic
style (35%) and socio-emotional traits (27%), supporting the idea that
intelligence is not sufficient to pass the Turing Test. Participants'
demographics, including education and familiarity with LLMs, did not predict
detection rate, suggesting that even those who understand systems deeply and
interact with them frequently may be susceptible to deception. Despite known
limitations as a test of intelligence, we argue that the Turing Test continues
to be relevant as an assessment of naturalistic communication and deception. AI
models with the ability to masquerade as humans could have widespread societal
consequences, and we analyse the effectiveness of different strategies and
criteria for judging humanlikeness.
6541cc6041676ceaa2ce8966
Cameron Jones
claimed_verified
2023-11-02T16:14:59.492Z
642b567d5df44ff24544e584
/avatars/f770958111b505232053cd415796dbe6.svg
Cameron
user
camrobjones
6541cc6041676ceaa2ce8967
Benjamin Bergen
claimed_verified
2025-04-08T07:03:33.792Z
656914264b8d35447adc5314
/avatars/04bcaf16a045cdc28e7503ecd5e7311d.svg
Ben Bergen
user
bkbergen
Does GPT-4 Pass the Turing Test?
We evaluated GPT-4 in a public online Turing Test. The best-performing GPT-4
prompt passed in 41% of games, outperforming baselines set by ELIZA (27%) and
GPT-3.5 (14%), but falling short of chance and the baseline set by human
participants (63%). Participants' decisions were based mainly on linguistic
style (35%) and socio-emotional traits (27%), supporting the idea that
intelligence is not sufficient to pass the Turing Test. Participants'
demographics, including education and familiarity with LLMs, did not predict
detection rate, suggesting that even those who understand systems deeply and
interact with them frequently may be susceptible to deception. Despite known
limitations as a test of intelligence, we argue that the Turing Test continues
to be relevant as an assessment of naturalistic communication and deception. AI
models with the ability to masquerade as humans could have widespread societal
consequences, and we analyse the effectiveness of different strategies and
criteria for judging humanlikeness.
GPT-4 performed better than GPT-3.5 and ELIZA in a public online Turing Test but failed to reach human-level performance, highlighting the importance of linguistic style and socio-emotional traits in human detection.
GPT-4 Turing Test GPT-3.5 ELIZA linguistic style socio-emotional traits naturalistic communication deception