title: VRAG-RL: Empower Vision-Perception-Based RAG for Visually Rich
    Information Understanding via Iterative Reasoning with Reinforcement Learning
summary: Effectively retrieving, reasoning and understanding visually rich information
  remains a challenge for RAG methods. Traditional text-based methods cannot
  handle visual-related information. On the other hand, current vision-based RAG
  approaches are often limited by fixed pipelines and frequently struggle to
  reason effectively due to the insufficient activation of the fundamental
  capabilities of models. As RL has been proven to be beneficial for model
  reasoning, we introduce VRAG-RL, a novel RL framework tailored for complex
  reasoning across visually rich information. With this framework, VLMs interact
  with search engines, autonomously sampling single-turn or multi-turn reasoning
  trajectories with the help of visual perception tokens and undergoing continual
  optimization based on these samples. Our approach highlights key limitations of
  RL in RAG domains: (i) Prior Multi-modal RAG approaches tend to merely
  incorporate images into the context, leading to insufficient reasoning token
  allocation and neglecting visual-specific perception; and (ii) When models
  interact with search engines, their queries often fail to retrieve relevant
  information due to the inability to articulate requirements, thereby leading to
  suboptimal performance. To address these challenges, we define an action space
  tailored for visually rich inputs, with actions including cropping and scaling,
  allowing the model to gather information from a coarse-to-fine perspective.
  Furthermore, to bridge the gap between users' original inquiries and the
  retriever, we employ a simple yet effective reward that integrates query
  rewriting and retrieval performance with a model-based reward. Our VRAG-RL
  optimizes VLMs for RAG tasks using specially designed RL strategies, aligning
  the model with real-world applications. The code is available at
  https://github.com/Alibaba-NLP/VRAG{https://github.com/Alibaba-NLP/VRAG}.
paper_authors._id: 6837ed297d00cf0a04677bc1
paper_authors.name: Qiuchen Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-29T09:40:19.039Z
paper_authors._id: 657429d833e5a4bf5b278615
paper_authors.avatarUrl: /avatars/ed7e28c1b9a7bed1cad864c992cdcc69.svg
paper_authors.fullname: QiuchenWang
paper_authors.type: user
paper_authors.user: autumncc
paper_authors._id: 6837ed297d00cf0a04677bc2
paper_authors.name: Ruixue Ding
paper_authors._id: 6837ed297d00cf0a04677bc3
paper_authors.name: Yu Zeng
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-29T07:41:42.906Z
paper_authors._id: 665d652e0f35c005de892108
paper_authors.avatarUrl: /avatars/240bebdc7fdc6d50719c65de0e3cf1cd.svg
paper_authors.fullname: Yu Zeng
paper_authors.type: user
paper_authors.user: YuZeng260
paper_authors._id: 6837ed297d00cf0a04677bc4
paper_authors.name: Zehui Chen
paper_authors._id: 6837ed297d00cf0a04677bc5
paper_authors.name: Lin Chen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-29T07:41:44.983Z
paper_authors._id: 64b02ec0e5000ae8a572ced5
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64b02ec0e5000ae8a572ced5/6ifLntBU2ICQK7SW8WxKU.png
paper_authors.fullname: Lin Chen
paper_authors.type: user
paper_authors.user: Lin-Chen
paper_authors._id: 6837ed297d00cf0a04677bc6
paper_authors.name: Shihang Wang
paper_authors._id: 6837ed297d00cf0a04677bc7
paper_authors.name: Pengjun Xie
paper_authors._id: 6837ed297d00cf0a04677bc8
paper_authors.name: Fei Huang
paper_authors._id: 6837ed297d00cf0a04677bc9
paper_authors.name: Feng Zhao
paper_title: VRAG-RL: Empower Vision-Perception-Based RAG for Visually Rich
    Information Understanding via Iterative Reasoning with Reinforcement Learning
paper_summary: Effectively retrieving, reasoning and understanding visually rich information
  remains a challenge for RAG methods. Traditional text-based methods cannot
  handle visual-related information. On the other hand, current vision-based RAG
  approaches are often limited by fixed pipelines and frequently struggle to
  reason effectively due to the insufficient activation of the fundamental
  capabilities of models. As RL has been proven to be beneficial for model
  reasoning, we introduce VRAG-RL, a novel RL framework tailored for complex
  reasoning across visually rich information. With this framework, VLMs interact
  with search engines, autonomously sampling single-turn or multi-turn reasoning
  trajectories with the help of visual perception tokens and undergoing continual
  optimization based on these samples. Our approach highlights key limitations of
  RL in RAG domains: (i) Prior Multi-modal RAG approaches tend to merely
  incorporate images into the context, leading to insufficient reasoning token
  allocation and neglecting visual-specific perception; and (ii) When models
  interact with search engines, their queries often fail to retrieve relevant
  information due to the inability to articulate requirements, thereby leading to
  suboptimal performance. To address these challenges, we define an action space
  tailored for visually rich inputs, with actions including cropping and scaling,
  allowing the model to gather information from a coarse-to-fine perspective.
  Furthermore, to bridge the gap between users' original inquiries and the
  retriever, we employ a simple yet effective reward that integrates query
  rewriting and retrieval performance with a model-based reward. Our VRAG-RL
  optimizes VLMs for RAG tasks using specially designed RL strategies, aligning
  the model with real-world applications. The code is available at
  https://github.com/Alibaba-NLP/VRAG{https://github.com/Alibaba-NLP/VRAG}.
paper_ai_summary: VRAG-RL, a reinforcement learning framework, enhances reasoning and visual information handling in RAG methods by integrating visual perception tokens and employing specialized action spaces and rewards.
paper_ai_keywords: reinforcement learning VRAG-RL VLMs search engines visually rich information reasoning trajectories visual perception tokens action space query rewriting retrieval performance model-based reward