A Picture Is Worth a Graph: A Blueprint Debate Paradigm for Multimodal Reasoning
<|reference_start|>A Picture Is Worth a Graph: A Blueprint Debate Paradigm for Multimodal Reasoning: This paper presents a pilot study aimed at introducing multi-agent debate into multimodal reasoning. The study addresses two key challenges: the trivialization of opinions resulting from excessive summarization and the diversion of focus caused by distractor concepts introduced from images. These challenges stem from the inductive (bottom-up) nature of existing debating schemes. To address the issue, we propose a deductive (top-down) debating approach called Blueprint Debate on Graphs (BDoG). In BDoG, debates are confined to a blueprint graph to prevent opinion trivialization through world-level summarization. Moreover, by storing evidence in branches within the graph, BDoG mitigates distractions caused by frequent but irrelevant concepts. Extensive experiments validate that BDoG is able to achieve state-of-the-art results in ScienceQA and MMBench with significant improvements over previous methods. The source code can be accessed at https://github.com/thecharm/BDoG.<|reference_end|>
@article{zheng2024a,
    title={A Picture Is Worth a Graph: A Blueprint Debate Paradigm for Multimodal
  Reasoning},
    author={Changmeng Zheng, Dayong Liang, Wengyu Zhang, Xiao-Yong Wei, Tat-Seng
  Chua, Qing Li},
    journal={arXiv preprint arXiv:2403.14972},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.14972},
    primaryClass={cs.AI cs.CL cs.MA cs.MM}
}