Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated
  Text
Detecting text generated by modern large language models is thought to be
hard, as both LLMs and humans can exhibit a wide range of complex behaviors.
However, we find that a score based on contrasting two closely related language
models is highly accurate at separating human-generated and machine-generated
text. Based on this mechanism, we propose a novel LLM detector that only
requires simple calculations using a pair of pre-trained LLMs. The method,
called Binoculars, achieves state-of-the-art accuracy without any training
data. It is capable of spotting machine text from a range of modern LLMs
without any model-specific modifications. We comprehensively evaluate
Binoculars on a number of text sources and in varied situations. Over a wide
range of document types, Binoculars detects over 90% of generated samples from
ChatGPT (and other LLMs) at a false positive rate of 0.01%, despite not being
trained on any ChatGPT data.
65af32e41edab235a1f3a9e0
Abhimanyu Hans
admin_assigned
2024-01-23T09:49:25.517Z
6489c06c08b6a836a6dbd6aa
/avatars/6ed1928992fc9d24bad0f12ebd2d84fe.svg
Abhimanyu Hans
user
ahans1
65af32e41edab235a1f3a9e1
Avi Schwarzschild
admin_assigned
2024-01-23T09:49:32.749Z
6570917c0ea91e592aff0b8c
/avatars/529e9713e6ac835e11599ea7070a9603.svg
Avi Schwarzschild
user
schwarzschild
65af32e41edab235a1f3a9e2
Valeriia Cherepanova
admin_assigned
2024-01-23T09:49:39.483Z
64c2ae06a2f9706201409fef
/avatars/fb051d567411b88c7c0651051948e1d8.svg
Valeriia Cherepanova
user
vcherepanova
65af32e41edab235a1f3a9e3
Hamid Kazemi
admin_assigned
2024-04-15T16:42:46.467Z
64f1f9612820a6f1b9e19edd
https://cdn-avatars.huggingface.co/v1/production/uploads/64f1f9612820a6f1b9e19edd/hSmhLsczKR1t5jxHwEaE_.jpeg
Hamid Kazemi
user
seyedhamidreza
65af32e41edab235a1f3a9e4
Aniruddha Saha
admin_assigned
2024-01-23T09:49:49.385Z
64178c29fff753e7c156a5da
https://cdn-avatars.huggingface.co/v1/production/uploads/1679264854378-64178c29fff753e7c156a5da.jpeg
Aniruddha Saha
user
ani0075saha
65af32e41edab235a1f3a9e5
Micah Goldblum
65af32e41edab235a1f3a9e6
Jonas Geiping
admin_assigned
2024-01-23T09:49:59.164Z
63d86dbf3130cadcaf8bdd11
/avatars/29d79a0c6dcec01111ef192fecd0fa7a.svg
Jonas Geiping
user
JonasGeiping
65af32e41edab235a1f3a9e7
Tom Goldstein
admin_assigned
2024-01-23T09:50:05.842Z
6381ca7d65dc156aba0b933d
/avatars/84dfdca8e1cd6fbf50d6fb2a6f1b488d.svg
Tom Goldstein
user
tomgoldstein
Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated
  Text
Detecting text generated by modern large language models is thought to be
hard, as both LLMs and humans can exhibit a wide range of complex behaviors.
However, we find that a score based on contrasting two closely related language
models is highly accurate at separating human-generated and machine-generated
text. Based on this mechanism, we propose a novel LLM detector that only
requires simple calculations using a pair of pre-trained LLMs. The method,
called Binoculars, achieves state-of-the-art accuracy without any training
data. It is capable of spotting machine text from a range of modern LLMs
without any model-specific modifications. We comprehensively evaluate
Binoculars on a number of text sources and in varied situations. Over a wide
range of document types, Binoculars detects over 90% of generated samples from
ChatGPT (and other LLMs) at a false positive rate of 0.01%, despite not being
trained on any ChatGPT data.
A novel LLM detector using contrasting scores from two pre-trained models achieves high accuracy in identifying machine-generated text without any training data.
LLMs language models Binoculars pre-trained LLMs