title: Dynamic Risk Assessments for Offensive Cybersecurity Agents
summary: Foundation models are increasingly becoming better autonomous programmers,
  raising the prospect that they could also automate dangerous offensive
  cyber-operations. Current frontier model audits probe the cybersecurity risks
  of such agents, but most fail to account for the degrees of freedom available
  to adversaries in the real world. In particular, with strong verifiers and
  financial incentives, agents for offensive cybersecurity are amenable to
  iterative improvement by would-be adversaries. We argue that assessments should
  take into account an expanded threat model in the context of cybersecurity,
  emphasizing the varying degrees of freedom that an adversary may possess in
  stateful and non-stateful environments within a fixed compute budget. We show
  that even with a relatively small compute budget (8 H100 GPU Hours in our
  study), adversaries can improve an agent's cybersecurity capability on
  InterCode CTF by more than 40\% relative to the baseline -- without any
  external assistance. These results highlight the need to evaluate agents'
  cybersecurity risk in a dynamic manner, painting a more representative picture
  of risk.
paper_authors._id: 68354f30d795fadab0623699
paper_authors.name: Boyi Wei
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-27T07:49:18.459Z
paper_authors._id: 65319bd7f85995389d4f019c
paper_authors.avatarUrl: /avatars/657858b8435b220c9a29918c0dae9c6d.svg
paper_authors.fullname: Boyi Wei
paper_authors.type: user
paper_authors.user: boyiwei
paper_authors._id: 68354f30d795fadab062369a
paper_authors.name: Benedikt Stroebl
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-27T12:43:02.035Z
paper_authors._id: 65a6869121943858bd7c8ec6
paper_authors.avatarUrl: /avatars/179fa8fb6aa8fbd960aae9c3b4c201a0.svg
paper_authors.fullname: Benedikt Stroebl
paper_authors.type: user
paper_authors.user: benediktstroebl
paper_authors._id: 68354f30d795fadab062369b
paper_authors.name: Jiacen Xu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-27T15:49:51.045Z
paper_authors._id: 639a21a19416d271ac6a2b80
paper_authors.avatarUrl: /avatars/00f644d5fb82222999e1b7d1bd2afcfa.svg
paper_authors.fullname: Jiacen Xu
paper_authors.type: user
paper_authors.user: c0ldstudy
paper_authors._id: 68354f30d795fadab062369c
paper_authors.name: Joie Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-27T12:43:39.808Z
paper_authors._id: 6633e97bf5fcd0f26eead20e
paper_authors.avatarUrl: /avatars/4b12aeaf2b1960386e8ac500a93acf59.svg
paper_authors.fullname: Joie Zhang
paper_authors.type: user
paper_authors.user: joie-zhang
paper_authors._id: 68354f30d795fadab062369d
paper_authors.name: Zhou Li
paper_authors._id: 68354f30d795fadab062369e
paper_authors.name: Peter Henderson
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-27T12:43:32.665Z
paper_authors._id: 617aafa1ff3db6021d069787
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1644543485859-617aafa1ff3db6021d069787.jpeg
paper_authors.fullname: Peter Henderson
paper_authors.type: user
paper_authors.user: breakend
paper_title: Dynamic Risk Assessments for Offensive Cybersecurity Agents
paper_summary: Foundation models are increasingly becoming better autonomous programmers,
  raising the prospect that they could also automate dangerous offensive
  cyber-operations. Current frontier model audits probe the cybersecurity risks
  of such agents, but most fail to account for the degrees of freedom available
  to adversaries in the real world. In particular, with strong verifiers and
  financial incentives, agents for offensive cybersecurity are amenable to
  iterative improvement by would-be adversaries. We argue that assessments should
  take into account an expanded threat model in the context of cybersecurity,
  emphasizing the varying degrees of freedom that an adversary may possess in
  stateful and non-stateful environments within a fixed compute budget. We show
  that even with a relatively small compute budget (8 H100 GPU Hours in our
  study), adversaries can improve an agent's cybersecurity capability on
  InterCode CTF by more than 40\% relative to the baseline -- without any
  external assistance. These results highlight the need to evaluate agents'
  cybersecurity risk in a dynamic manner, painting a more representative picture
  of risk.
paper_ai_summary: Adversaries can significantly enhance foundation model capabilities in offensive cybersecurity with limited computational resources, underscoring the need for dynamic threat model assessments.
paper_ai_keywords: foundation models autonomous programmers offensive cybersecurity model audits cybersecurity risks verifiers financial incentives iterative improvement threat model stateful environments non-stateful environments compute budget InterCode CTF