title: Evaluating Frontier Models for Dangerous Capabilities
summary: To understand the risks posed by a new AI system, we must understand what it
  can and cannot do. Building on prior work, we introduce a programme of new
  "dangerous capability" evaluations and pilot them on Gemini 1.0 models. Our
  evaluations cover four areas: (1) persuasion and deception; (2) cyber-security;
  (3) self-proliferation; and (4) self-reasoning. We do not find evidence of
  strong dangerous capabilities in the models we evaluated, but we flag early
  warning signs. Our goal is to help advance a rigorous science of dangerous
  capability evaluation, in preparation for future models.
paper_authors._id: 65fb9eefae01a24b40e2c49c
paper_authors.name: Mary Phuong
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2024-04-26T15:32:57.340Z
paper_authors._id: 662bc91a616128914af09df6
paper_authors.avatarUrl: /avatars/ccfc7cff9e5b5f10b7555e1887d66dd6.svg
paper_authors.fullname: Mary Phuong
paper_authors.type: user
paper_authors.user: mphuong26
paper_authors._id: 65fb9eefae01a24b40e2c49d
paper_authors.name: Matthew Aitchison
paper_authors._id: 65fb9eefae01a24b40e2c49e
paper_authors.name: Elliot Catt
paper_authors._id: 65fb9eefae01a24b40e2c49f
paper_authors.name: Sarah Cogan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-21T11:47:36.474Z
paper_authors._id: 644165eb66a62c605c957d6d
paper_authors.avatarUrl: /avatars/87aa072c371cc87d1d9e3c18169a2e3d.svg
paper_authors.fullname: Sarah Cogan
paper_authors.type: user
paper_authors.user: sarahcogan
paper_authors._id: 65fb9eefae01a24b40e2c4a0
paper_authors.name: Alexandre Kaskasoli
paper_authors._id: 65fb9eefae01a24b40e2c4a1
paper_authors.name: Victoria Krakovna
paper_authors._id: 65fb9eefae01a24b40e2c4a2
paper_authors.name: David Lindner
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-21T11:47:53.605Z
paper_authors._id: 64aef034710608ee1a02b1ac
paper_authors.avatarUrl: /avatars/0f8449bf7ee2a908962502ec863155be.svg
paper_authors.fullname: David Lindner
paper_authors.type: user
paper_authors.user: 1obextiopo
paper_authors._id: 65fb9eefae01a24b40e2c4a3
paper_authors.name: Matthew Rahtz
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-21T11:48:00.324Z
paper_authors._id: 63cd53841c8a5d1d7d772d09
paper_authors.avatarUrl: /avatars/dab344bf89655a450ab5d4ad39fcf7d2.svg
paper_authors.fullname: Matthew Rahtz
paper_authors.type: user
paper_authors.user: mrahtz
paper_authors._id: 65fb9eefae01a24b40e2c4a4
paper_authors.name: Yannis Assael
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-21T11:48:07.770Z
paper_authors._id: 62cdbdcdc589e4a9e24088d2
paper_authors.avatarUrl: /avatars/db976d12e5531aeba7c8167fe1a9b88b.svg
paper_authors.fullname: Yannis Assael
paper_authors.type: user
paper_authors.user: iassael
paper_authors._id: 65fb9eefae01a24b40e2c4a5
paper_authors.name: Sarah Hodkinson
paper_authors._id: 65fb9eefae01a24b40e2c4a6
paper_authors.name: Heidi Howard
paper_authors._id: 65fb9eefae01a24b40e2c4a7
paper_authors.name: Tom Lieberum
paper_authors._id: 65fb9eefae01a24b40e2c4a8
paper_authors.name: Ramana Kumar
paper_authors._id: 65fb9eefae01a24b40e2c4a9
paper_authors.name: Maria Abi Raad
paper_authors._id: 65fb9eefae01a24b40e2c4aa
paper_authors.name: Albert Webson
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-21T11:48:56.710Z
paper_authors._id: 617ada70ff3db6021d069798
paper_authors.avatarUrl: /avatars/f14c59fd56a49dd9a925285a4fb66fff.svg
paper_authors.fullname: Albert Webson
paper_authors.type: user
paper_authors.user: awebson
paper_authors._id: 65fb9eefae01a24b40e2c4ab
paper_authors.name: Lewis Ho
paper_authors._id: 65fb9eefae01a24b40e2c4ac
paper_authors.name: Sharon Lin
paper_authors._id: 65fb9eefae01a24b40e2c4ad
paper_authors.name: Sebastian Farquhar
paper_authors._id: 65fb9eefae01a24b40e2c4ae
paper_authors.name: Marcus Hutter
paper_authors._id: 65fb9eefae01a24b40e2c4af
paper_authors.name: Gregoire Deletang
paper_authors._id: 65fb9eefae01a24b40e2c4b0
paper_authors.name: Anian Ruoss
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-21T11:49:40.175Z
paper_authors._id: 65c51758522ee3a4edd917d0
paper_authors.avatarUrl: /avatars/2ea3b64b5eeb34e2bfe821e4210f1b3e.svg
paper_authors.fullname: Anian Ruoss
paper_authors.type: user
paper_authors.user: anianruoss
paper_authors._id: 65fb9eefae01a24b40e2c4b1
paper_authors.name: Seliem El-Sayed
paper_authors._id: 65fb9eefae01a24b40e2c4b2
paper_authors.name: Sasha Brown
paper_authors._id: 65fb9eefae01a24b40e2c4b3
paper_authors.name: Anca Dragan
paper_authors._id: 65fb9eefae01a24b40e2c4b4
paper_authors.name: Rohin Shah
paper_authors._id: 65fb9eefae01a24b40e2c4b5
paper_authors.name: Allan Dafoe
paper_authors._id: 65fb9eefae01a24b40e2c4b6
paper_authors.name: Toby Shevlane
paper_title: Evaluating Frontier Models for Dangerous Capabilities
paper_summary: To understand the risks posed by a new AI system, we must understand what it
  can and cannot do. Building on prior work, we introduce a programme of new
  "dangerous capability" evaluations and pilot them on Gemini 1.0 models. Our
  evaluations cover four areas: (1) persuasion and deception; (2) cyber-security;
  (3) self-proliferation; and (4) self-reasoning. We do not find evidence of
  strong dangerous capabilities in the models we evaluated, but we flag early
  warning signs. Our goal is to help advance a rigorous science of dangerous
  capability evaluation, in preparation for future models.
paper_ai_keywords: supervous capability persuasion and deception cyber-security self-proliferation self-reasoning Gemini 1.0 dangerous capabilities semi-rigorous science