title: Revisiting Multi-Agent Debate as Test-Time Scaling: A Systematic Study
    of Conditional Effectiveness
summary: The remarkable growth in large language model (LLM) capabilities has spurred
  exploration into multi-agent systems, with debate frameworks emerging as a
  promising avenue for enhanced problem-solving. These multi-agent debate (MAD)
  approaches, where agents collaboratively present, critique, and refine
  arguments, potentially offer improved reasoning, robustness, and diverse
  perspectives over monolithic models. Despite prior studies leveraging MAD, a
  systematic understanding of its effectiveness compared to self-agent methods,
  particularly under varying conditions, remains elusive. This paper seeks to
  fill this gap by conceptualizing MAD as a test-time computational scaling
  technique, distinguished by collaborative refinement and diverse exploration
  capabilities. We conduct a comprehensive empirical investigation comparing MAD
  with strong self-agent test-time scaling baselines on mathematical reasoning
  and safety-related tasks. Our study systematically examines the influence of
  task difficulty, model scale, and agent diversity on MAD's performance. Key
  findings reveal that, for mathematical reasoning, MAD offers limited advantages
  over self-agent scaling but becomes more effective with increased problem
  difficulty and decreased model capability, while agent diversity shows little
  benefit. Conversely, for safety tasks, MAD's collaborative refinement can
  increase vulnerability, but incorporating diverse agent configurations
  facilitates a gradual reduction in attack success through the collaborative
  refinement process. We believe our findings provide critical guidance for the
  future development of more effective and strategically deployed MAD systems.
paper_authors._id: 68390972be7fee960c0b07ee
paper_authors.name: Yongjin Yang
paper_authors._id: 68390972be7fee960c0b07ef
paper_authors.name: Euiin Yi
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-02T07:47:31.722Z
paper_authors._id: 66027cead56cd39a44992631
paper_authors.avatarUrl: /avatars/3c8b679bc72f30c547782579e9592bb1.svg
paper_authors.fullname: Euiin Yi
paper_authors.type: user
paper_authors.user: euiin
paper_authors._id: 68390972be7fee960c0b07f0
paper_authors.name: Jongwoo Ko
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-30T06:54:24.221Z
paper_authors._id: 64b7628af902508f0d7ae112
paper_authors.avatarUrl: /avatars/83c155254486e80c1dfd14676fdf9215.svg
paper_authors.fullname: Jongwoo Ko
paper_authors.type: user
paper_authors.user: jongwooko
paper_authors._id: 68390972be7fee960c0b07f1
paper_authors.name: Kimin Lee
paper_authors._id: 68390972be7fee960c0b07f2
paper_authors.name: Zhijing Jin
paper_authors._id: 68390972be7fee960c0b07f3
paper_authors.name: Se-Young Yun
paper_title: Revisiting Multi-Agent Debate as Test-Time Scaling: A Systematic Study
    of Conditional Effectiveness
paper_summary: The remarkable growth in large language model (LLM) capabilities has spurred
  exploration into multi-agent systems, with debate frameworks emerging as a
  promising avenue for enhanced problem-solving. These multi-agent debate (MAD)
  approaches, where agents collaboratively present, critique, and refine
  arguments, potentially offer improved reasoning, robustness, and diverse
  perspectives over monolithic models. Despite prior studies leveraging MAD, a
  systematic understanding of its effectiveness compared to self-agent methods,
  particularly under varying conditions, remains elusive. This paper seeks to
  fill this gap by conceptualizing MAD as a test-time computational scaling
  technique, distinguished by collaborative refinement and diverse exploration
  capabilities. We conduct a comprehensive empirical investigation comparing MAD
  with strong self-agent test-time scaling baselines on mathematical reasoning
  and safety-related tasks. Our study systematically examines the influence of
  task difficulty, model scale, and agent diversity on MAD's performance. Key
  findings reveal that, for mathematical reasoning, MAD offers limited advantages
  over self-agent scaling but becomes more effective with increased problem
  difficulty and decreased model capability, while agent diversity shows little
  benefit. Conversely, for safety tasks, MAD's collaborative refinement can
  increase vulnerability, but incorporating diverse agent configurations
  facilitates a gradual reduction in attack success through the collaborative
  refinement process. We believe our findings provide critical guidance for the
  future development of more effective and strategically deployed MAD systems.
paper_ai_summary: Multi-agent debate systems offer variable benefits over self-agent approaches in mathematical reasoning and safety tasks, with performance influenced by task difficulty, model scale, and agent diversity.
paper_ai_keywords: large language model multi-agent systems debate frameworks multi-agent debate collaborative refinement diverse exploration self-agent methods test-time computational scaling mathematical reasoning safety-related tasks agent diversity collaborative refinement process