DeepResearch Bench: A Comprehensive Benchmark for Deep Research Agents
Deep Research Agents are a prominent category of LLM-based agents. By
autonomously orchestrating multistep web exploration, targeted retrieval, and
higher-order synthesis, they transform vast amounts of online information into
analyst-grade, citation-rich reports--compressing hours of manual desk research
into minutes. However, a comprehensive benchmark for systematically evaluating
the capabilities of these agents remains absent. To bridge this gap, we present
DeepResearch Bench, a benchmark consisting of 100 PhD-level research tasks,
each meticulously crafted by domain experts across 22 distinct fields.
Evaluating DRAs is inherently complex and labor-intensive. We therefore propose
two novel methodologies that achieve strong alignment with human judgment. The
first is a reference-based method with adaptive criteria to assess the quality
of generated research reports. The other framework is introduced to evaluate
DRA's information retrieval and collection capabilities by assessing its
effective citation count and overall citation accuracy. We have open-sourced
DeepResearch Bench and key components of these frameworks at
https://github.com/Ayanami0730/deep_research_bench to accelerate the
development of practical LLM-based agents.
684ff5051d9b438aa3957a7f
Mingxuan Du
claimed_verified
2025-06-16T12:56:12.158Z
646dbba74ad7f907279dd486
/avatars/fe2b95e9a55711164e9624e1d15e0af2.svg
Mingxuan Du
user
Ayanami0730
684ff5051d9b438aa3957a80
Benfeng Xu
684ff5051d9b438aa3957a81
Chiwei Zhu
claimed_verified
2025-06-16T12:56:09.702Z
663b22a80966eef8686aadaf
https://cdn-avatars.huggingface.co/v1/production/uploads/663b22a80966eef8686aadaf/iBzyQTyGZKf33RPVIFh9a.jpeg
Chiwei Zhu
user
IgnoraZ
684ff5051d9b438aa3957a82
Xiaorui Wang
684ff5051d9b438aa3957a83
Zhendong Mao
DeepResearch Bench: A Comprehensive Benchmark for Deep Research Agents
Deep Research Agents are a prominent category of LLM-based agents. By
autonomously orchestrating multistep web exploration, targeted retrieval, and
higher-order synthesis, they transform vast amounts of online information into
analyst-grade, citation-rich reports--compressing hours of manual desk research
into minutes. However, a comprehensive benchmark for systematically evaluating
the capabilities of these agents remains absent. To bridge this gap, we present
DeepResearch Bench, a benchmark consisting of 100 PhD-level research tasks,
each meticulously crafted by domain experts across 22 distinct fields.
Evaluating DRAs is inherently complex and labor-intensive. We therefore propose
two novel methodologies that achieve strong alignment with human judgment. The
first is a reference-based method with adaptive criteria to assess the quality
of generated research reports. The other framework is introduced to evaluate
DRA's information retrieval and collection capabilities by assessing its
effective citation count and overall citation accuracy. We have open-sourced
DeepResearch Bench and key components of these frameworks at
https://github.com/Ayanami0730/deep_research_bench to accelerate the
development of practical LLM-based agents.
DeepResearch Bench offers a benchmark framework to evaluate the capabilities of Deep Research Agents in terms of research quality and information retrieval accuracy across multiple fields.
Deep Research Agents LLM-based agents multistep web exploration targeted retrieval higher-order synthesis PhD-level research tasks reference-based method effective citation count citation accuracy