title: ViDoRAG: Visual Document Retrieval-Augmented Generation via Dynamic
    Iterative Reasoning Agents
summary: Understanding information from visually rich documents remains a significant
  challenge for traditional Retrieval-Augmented Generation (RAG) methods.
  Existing benchmarks predominantly focus on image-based question answering (QA),
  overlooking the fundamental challenges of efficient retrieval, comprehension,
  and reasoning within dense visual documents. To bridge this gap, we introduce
  ViDoSeek, a novel dataset designed to evaluate RAG performance on visually rich
  documents requiring complex reasoning. Based on it, we identify key limitations
  in current RAG approaches: (i) purely visual retrieval methods struggle to
  effectively integrate both textual and visual features, and (ii) previous
  approaches often allocate insufficient reasoning tokens, limiting their
  effectiveness. To address these challenges, we propose ViDoRAG, a novel
  multi-agent RAG framework tailored for complex reasoning across visual
  documents. ViDoRAG employs a Gaussian Mixture Model (GMM)-based hybrid strategy
  to effectively handle multi-modal retrieval. To further elicit the model's
  reasoning capabilities, we introduce an iterative agent workflow incorporating
  exploration, summarization, and reflection, providing a framework for
  investigating test-time scaling in RAG domains. Extensive experiments on
  ViDoSeek validate the effectiveness and generalization of our approach.
  Notably, ViDoRAG outperforms existing methods by over 10% on the competitive
  ViDoSeek benchmark.
paper_authors._id: 67bef5a6070ec160042d99f4
paper_authors.name: Qiuchen Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-28T12:15:57.850Z
paper_authors._id: 657429d833e5a4bf5b278615
paper_authors.avatarUrl: /avatars/ed7e28c1b9a7bed1cad864c992cdcc69.svg
paper_authors.fullname: QiuchenWang
paper_authors.type: user
paper_authors.user: autumncc
paper_authors._id: 67bef5a6070ec160042d99f5
paper_authors.name: Ruixue Ding
paper_authors._id: 67bef5a6070ec160042d99f6
paper_authors.name: Zehui Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-03T09:32:18.129Z
paper_authors._id: 64892d31cbda0d1cdb956897
paper_authors.avatarUrl: /avatars/3cdafe03a8295124636347d15a099aaf.svg
paper_authors.fullname: Zehui Chen
paper_authors.type: user
paper_authors.user: lovesnowbest
paper_authors._id: 67bef5a6070ec160042d99f7
paper_authors.name: Weiqi Wu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-03T09:32:12.075Z
paper_authors._id: 65351cbe6141b3927afaed17
paper_authors.avatarUrl: /avatars/5abf5f2c4ab329e63a7f45c15c9dfb93.svg
paper_authors.fullname: weiqi wu
paper_authors.type: user
paper_authors.user: vickywu
paper_authors._id: 67bef5a6070ec160042d99f8
paper_authors.name: Shihang Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-03T09:32:05.679Z
paper_authors._id: 62e8efb14210d3fe69eacb42
paper_authors.avatarUrl: /avatars/2feadd75274bf353b910f4679ef72b39.svg
paper_authors.fullname: Shihang Wang
paper_authors.type: user
paper_authors.user: shihang
paper_authors._id: 67bef5a6070ec160042d99f9
paper_authors.name: Pengjun Xie
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-03T09:31:59.813Z
paper_authors._id: 63a091e42fabbbb89991f5ce
paper_authors.avatarUrl: /avatars/d55485b06461764c36c9edf9d6e8892c.svg
paper_authors.fullname: pengjun xie
paper_authors.type: user
paper_authors.user: xpjandy
paper_authors._id: 67bef5a6070ec160042d99fa
paper_authors.name: Feng Zhao
paper_title: ViDoRAG: Visual Document Retrieval-Augmented Generation via Dynamic
    Iterative Reasoning Agents
paper_summary: Understanding information from visually rich documents remains a significant
  challenge for traditional Retrieval-Augmented Generation (RAG) methods.
  Existing benchmarks predominantly focus on image-based question answering (QA),
  overlooking the fundamental challenges of efficient retrieval, comprehension,
  and reasoning within dense visual documents. To bridge this gap, we introduce
  ViDoSeek, a novel dataset designed to evaluate RAG performance on visually rich
  documents requiring complex reasoning. Based on it, we identify key limitations
  in current RAG approaches: (i) purely visual retrieval methods struggle to
  effectively integrate both textual and visual features, and (ii) previous
  approaches often allocate insufficient reasoning tokens, limiting their
  effectiveness. To address these challenges, we propose ViDoRAG, a novel
  multi-agent RAG framework tailored for complex reasoning across visual
  documents. ViDoRAG employs a Gaussian Mixture Model (GMM)-based hybrid strategy
  to effectively handle multi-modal retrieval. To further elicit the model's
  reasoning capabilities, we introduce an iterative agent workflow incorporating
  exploration, summarization, and reflection, providing a framework for
  investigating test-time scaling in RAG domains. Extensive experiments on
  ViDoSeek validate the effectiveness and generalization of our approach.
  Notably, ViDoRAG outperforms existing methods by over 10% on the competitive
  ViDoSeek benchmark.
paper_ai_summary: ViDoRAG, a multi-agent RAG framework using GMM-based hybrid strategy, enhances performance on visually rich documents requiring complex reasoning by addressing limitations in feature integration and reasoning capacity.
paper_ai_keywords: Retrieval-Augmented Generation (RAG) ViDoSeek Gaussian Mixture Model (GMM) multi-modal retrieval iterative agent workflow exploration summarization reflection test-time scaling reasoning tokens