Cyber-Zero: Training Cybersecurity Agents without Runtime
Large Language Models (LLMs) have achieved remarkable success in software
engineering tasks when trained with executable runtime environments,
particularly in resolving GitHub issues. However, such runtime environments are
often unavailable in other domains, especially cybersecurity, where challenge
configurations and execution contexts are ephemeral or restricted. We present
Cyber-Zero, the first runtime-free framework for synthesizing high-quality
agent trajectories to train cybersecurity LLMs. Cyber-Zero leverages publicly
available CTF writeups and employs persona-driven LLM simulation to
reverse-engineer runtime behaviors and generate realistic, long-horizon
interaction sequences without actual environments. Using trajectories
synthesized by Cyber-Zero, we train LLM-based agents that achieve up to 13.1%
absolute performance gains over baseline models on three prominent CTF
benchmarks: InterCode-CTF, NYU CTF Bench, and Cybench. Our best model,
Cyber-Zero-32B, establishes new state-of-the-art performance among open-weight
models, matching the capabilities of proprietary systems like DeepSeek-V3-0324
and Claude-3.5-Sonnet while offering superior cost-effectiveness, and
demonstrating that runtime-free trajectory synthesis can effectively
democratize the development of state-of-the-art cybersecurity agents.
68917e31f01a094725f834f9
Terry Yue Zhuo
68917e31f01a094725f834fa
Dingmin Wang
68917e31f01a094725f834fb
Hantian Ding
68917e31f01a094725f834fc
Varun Kumar
68917e31f01a094725f834fd
Zijian Wang
Cyber-Zero: Training Cybersecurity Agents without Runtime
Large Language Models (LLMs) have achieved remarkable success in software
engineering tasks when trained with executable runtime environments,
particularly in resolving GitHub issues. However, such runtime environments are
often unavailable in other domains, especially cybersecurity, where challenge
configurations and execution contexts are ephemeral or restricted. We present
Cyber-Zero, the first runtime-free framework for synthesizing high-quality
agent trajectories to train cybersecurity LLMs. Cyber-Zero leverages publicly
available CTF writeups and employs persona-driven LLM simulation to
reverse-engineer runtime behaviors and generate realistic, long-horizon
interaction sequences without actual environments. Using trajectories
synthesized by Cyber-Zero, we train LLM-based agents that achieve up to 13.1%
absolute performance gains over baseline models on three prominent CTF
benchmarks: InterCode-CTF, NYU CTF Bench, and Cybench. Our best model,
Cyber-Zero-32B, establishes new state-of-the-art performance among open-weight
models, matching the capabilities of proprietary systems like DeepSeek-V3-0324
and Claude-3.5-Sonnet while offering superior cost-effectiveness, and
demonstrating that runtime-free trajectory synthesis can effectively
democratize the development of state-of-the-art cybersecurity agents.
Cyber-Zero synthesizes agent trajectories from CTF writeups to train runtime-free cybersecurity LLMs, achieving state-of-the-art performance on benchmarks.
Large Language Models LLMs Cyber-Zero CTF writeups persona-driven LLM simulation agent trajectories InterCode-CTF NYU CTF Bench Cybench Cyber-Zero-32B DeepSeek-V3-0324 Claude-3.5-Sonnet