title: When AI Co-Scientists Fail: SPOT-a Benchmark for Automated Verification
    of Scientific Research
summary: Recent advances in large language models (LLMs) have fueled the vision of
  automated scientific discovery, often called AI Co-Scientists. To date, prior
  work casts these systems as generative co-authors responsible for crafting
  hypotheses, synthesizing code, or drafting manuscripts. In this work, we
  explore a complementary application: using LLMs as verifiers to automate the
  academic verification of scientific manuscripts. To that end, we
  introduce SPOT, a dataset of 83 published papers paired with 91 errors
  significant enough to prompt errata or retraction, cross-validated with actual
  authors and human annotators. Evaluating state-of-the-art LLMs on SPOT, we find
  that none surpasses 21.1\% recall or 6.1\% precision (o3 achieves the best
  scores, with all others near zero). Furthermore, confidence estimates are
  uniformly low, and across eight independent runs, models rarely rediscover the
  same errors, undermining their reliability. Finally, qualitative analysis with
  domain experts reveals that even the strongest models make mistakes resembling
  student-level misconceptions derived from misunderstandings. These findings
  highlight the substantial gap between current LLM capabilities and the
  requirements for dependable AI-assisted academic verification.
paper_authors._id: 682c11fe08d047591841ebf1
paper_authors.name: Guijin Son
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-20T08:40:55.774Z
paper_authors._id: 60d3e619b8448e1785bbda2a
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/60d3e619b8448e1785bbda2a/q2re5u1HNwsCCyIMtid_I.jpeg
paper_authors.fullname: GUIJIN SON
paper_authors.type: user
paper_authors.user: amphora
paper_authors._id: 682c11fe08d047591841ebf2
paper_authors.name: Jiwoo Hong
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-20T08:41:14.541Z
paper_authors._id: 6415c043486c7c9a5d151583
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6415c043486c7c9a5d151583/fUdYFh6iVh57swCkBEy-y.jpeg
paper_authors.fullname: Jiwoo Hong
paper_authors.type: user
paper_authors.user: JW17
paper_authors._id: 682c11fe08d047591841ebf3
paper_authors.name: Honglu Fan
paper_authors._id: 682c11fe08d047591841ebf4
paper_authors.name: Heejeong Nam
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-20T08:41:30.411Z
paper_authors._id: 659f9445d5c4ea912705aa4d
paper_authors.avatarUrl: /avatars/1d3297c3ccad48e5eb6c01e0640dc06d.svg
paper_authors.fullname: Heejeong Nam
paper_authors.type: user
paper_authors.user: HazelNam
paper_authors._id: 682c11fe08d047591841ebf5
paper_authors.name: Hyunwoo Ko
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-20T07:20:13.177Z
paper_authors._id: 63e087b6a98d931aa90c1b9c
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63e087b6a98d931aa90c1b9c/96c6IT3f1pWGLbRdRDB2U.png
paper_authors.fullname: Hyunwoo Ko
paper_authors.type: user
paper_authors.user: Cartinoe5930
paper_authors._id: 682c11fe08d047591841ebf6
paper_authors.name: Seungwon Lim
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-20T08:41:49.745Z
paper_authors._id: 63be1cd13b0665ad51d29c37
paper_authors.avatarUrl: /avatars/5acc9b9bbecac3d567e927e2d8667b00.svg
paper_authors.fullname: Seungwon Lim
paper_authors.type: user
paper_authors.user: sngwon
paper_authors._id: 682c11fe08d047591841ebf7
paper_authors.name: Jinyeop Song
paper_authors._id: 682c11fe08d047591841ebf8
paper_authors.name: Jinha Choi
paper_authors._id: 682c11fe08d047591841ebf9
paper_authors.name: Gonçalo Paulo
paper_authors._id: 682c11fe08d047591841ebfa
paper_authors.name: Youngjae Yu
paper_authors._id: 682c11fe08d047591841ebfb
paper_authors.name: Stella Biderman
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-07T05:49:57.112Z
paper_authors._id: 60347d3660e3dd96631c9093
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/60347d3660e3dd96631c9093/B3fuZer5N04tZIAYrLnz4.jpeg
paper_authors.fullname: Stella Biderman
paper_authors.type: user
paper_authors.user: stellaathena
paper_title: When AI Co-Scientists Fail: SPOT-a Benchmark for Automated Verification
    of Scientific Research
paper_summary: Recent advances in large language models (LLMs) have fueled the vision of
  automated scientific discovery, often called AI Co-Scientists. To date, prior
  work casts these systems as generative co-authors responsible for crafting
  hypotheses, synthesizing code, or drafting manuscripts. In this work, we
  explore a complementary application: using LLMs as verifiers to automate the
  academic verification of scientific manuscripts. To that end, we
  introduce SPOT, a dataset of 83 published papers paired with 91 errors
  significant enough to prompt errata or retraction, cross-validated with actual
  authors and human annotators. Evaluating state-of-the-art LLMs on SPOT, we find
  that none surpasses 21.1\% recall or 6.1\% precision (o3 achieves the best
  scores, with all others near zero). Furthermore, confidence estimates are
  uniformly low, and across eight independent runs, models rarely rediscover the
  same errors, undermining their reliability. Finally, qualitative analysis with
  domain experts reveals that even the strongest models make mistakes resembling
  student-level misconceptions derived from misunderstandings. These findings
  highlight the substantial gap between current LLM capabilities and the
  requirements for dependable AI-assisted academic verification.
paper_ai_summary: Evaluation of LLMs on an academic manuscript verification dataset (SPOT) shows poor recall, precision, and reliability, indicating significant limitations in current AI's ability to replace human verification in scientific research.
paper_ai_keywords: large language models LLMs AI Co-Scientists generative co-authors academic verification of scientific manuscripts SPOT confidence estimates qualitative analysis