title: Visual Agentic Reinforcement Fine-Tuning
summary: A key trend in Large Reasoning Models (e.g., OpenAI's o3) is the native
  agentic ability to use external tools such as web browsers for searching and
  writing/executing code for image manipulation to think with images. In the
  open-source research community, while significant progress has been made in
  language-only agentic abilities such as function calling and tool integration,
  the development of multi-modal agentic capabilities that involve truly thinking
  with images, and their corresponding benchmarks, are still less explored. This
  work highlights the effectiveness of Visual Agentic Reinforcement Fine-Tuning
  (Visual-ARFT) for enabling flexible and adaptive reasoning abilities for Large
  Vision-Language Models (LVLMs). With Visual-ARFT, open-source LVLMs gain the
  ability to browse websites for real-time information updates and write code to
  manipulate and analyze input images through cropping, rotation, and other image
  processing techniques. We also present a Multi-modal Agentic Tool Bench (MAT)
  with two settings (MAT-Search and MAT-Coding) designed to evaluate LVLMs'
  agentic search and coding abilities. Our experimental results demonstrate that
  Visual-ARFT outperforms its baseline by +18.6% F1 / +13.0% EM on MAT-Coding and
  +10.3% F1 / +8.7% EM on MAT-Search, ultimately surpassing GPT-4o. Visual-ARFT
  also achieves +29.3 F1% / +25.9% EM gains on existing multi-hop QA benchmarks
  such as 2Wiki and HotpotQA, demonstrating strong generalization capabilities.
  Our findings suggest that Visual-ARFT offers a promising path toward building
  robust and generalizable multimodal agents.
paper_authors._id: 682d7a2340a42d1538fada76
paper_authors.name: Ziyu Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-21T10:08:39.517Z
paper_authors._id: 66fe1334ff3ee1f7569fab6d
paper_authors.avatarUrl: /avatars/6868b1a545028a9b8bbded52490dc093.svg
paper_authors.fullname: ziyuliu
paper_authors.type: user
paper_authors.user: ziyuliu
paper_authors._id: 682d7a2340a42d1538fada77
paper_authors.name: Yuhang Zang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-21T08:39:42.962Z
paper_authors._id: 63859cf3b2906edaf83af9f0
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63859cf3b2906edaf83af9f0/mCgynEtoXdILvzyoPexii.png
paper_authors.fullname: Yuhang Zang
paper_authors.type: user
paper_authors.user: yuhangzang
paper_authors._id: 682d7a2340a42d1538fada78
paper_authors.name: Yushan Zou
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-21T10:08:45.972Z
paper_authors._id: 6671341b4eee852a8b25888f
paper_authors.avatarUrl: /avatars/635d1821bbc960b4ea845e606883eb16.svg
paper_authors.fullname: yushan zou
paper_authors.type: user
paper_authors.user: zyshan
paper_authors._id: 682d7a2340a42d1538fada79
paper_authors.name: Zijian Liang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-21T10:08:51.938Z
paper_authors._id: 652768eeb723b49e8c8865da
paper_authors.avatarUrl: /avatars/491e02da9ec81e439ccda8a181634bca.svg
paper_authors.fullname: Zijian Liang
paper_authors.type: user
paper_authors.user: steins1096
paper_authors._id: 682d7a2340a42d1538fada7a
paper_authors.name: Xiaoyi Dong
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-21T10:09:00.912Z
paper_authors._id: 67c0849ee08c178ef8d4e05c
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/mQ6VdnjZnRhb0H_waPclo.png
paper_authors.fullname: Xiaoyi Dong
paper_authors.type: user
paper_authors.user: sweetFruit
paper_authors._id: 682d7a2340a42d1538fada7b
paper_authors.name: Yuhang Cao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-21T10:09:22.645Z
paper_authors._id: 65000bef18830fabea469fdd
paper_authors.avatarUrl: /avatars/b320c77dfad039d9f9c54127f610d44f.svg
paper_authors.fullname: Cao Yuhang
paper_authors.type: user
paper_authors.user: yhcao
paper_authors._id: 682d7a2340a42d1538fada7c
paper_authors.name: Haodong Duan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-21T10:09:31.442Z
paper_authors._id: 63ee1379190ddd6214efd73a
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1676546883247-noauth.png
paper_authors.fullname: HAODONG DUAN
paper_authors.type: user
paper_authors.user: KennyUTC
paper_authors._id: 682d7a2340a42d1538fada7d
paper_authors.name: Dahua Lin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-21T10:09:37.103Z
paper_authors._id: 636317ed80c1a705a6eff396
paper_authors.avatarUrl: /avatars/3db090e101b916d9256d0d3e043db71d.svg
paper_authors.fullname: Dahua Lin
paper_authors.type: user
paper_authors.user: lindahua
paper_authors._id: 682d7a2340a42d1538fada7e
paper_authors.name: Jiaqi Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-21T08:39:45.391Z
paper_authors._id: 64b4eec4faa3181a5eab9c46
paper_authors.avatarUrl: /avatars/bcc9bf5cbf67546ad2b4c9ec8b96ac96.svg
paper_authors.fullname: Jiaqi Wang
paper_authors.type: user
paper_authors.user: myownskyW7
paper_title: Visual Agentic Reinforcement Fine-Tuning
paper_summary: A key trend in Large Reasoning Models (e.g., OpenAI's o3) is the native
  agentic ability to use external tools such as web browsers for searching and
  writing/executing code for image manipulation to think with images. In the
  open-source research community, while significant progress has been made in
  language-only agentic abilities such as function calling and tool integration,
  the development of multi-modal agentic capabilities that involve truly thinking
  with images, and their corresponding benchmarks, are still less explored. This
  work highlights the effectiveness of Visual Agentic Reinforcement Fine-Tuning
  (Visual-ARFT) for enabling flexible and adaptive reasoning abilities for Large
  Vision-Language Models (LVLMs). With Visual-ARFT, open-source LVLMs gain the
  ability to browse websites for real-time information updates and write code to
  manipulate and analyze input images through cropping, rotation, and other image
  processing techniques. We also present a Multi-modal Agentic Tool Bench (MAT)
  with two settings (MAT-Search and MAT-Coding) designed to evaluate LVLMs'
  agentic search and coding abilities. Our experimental results demonstrate that
  Visual-ARFT outperforms its baseline by +18.6% F1 / +13.0% EM on MAT-Coding and
  +10.3% F1 / +8.7% EM on MAT-Search, ultimately surpassing GPT-4o. Visual-ARFT
  also achieves +29.3 F1% / +25.9% EM gains on existing multi-hop QA benchmarks
  such as 2Wiki and HotpotQA, demonstrating strong generalization capabilities.
  Our findings suggest that Visual-ARFT offers a promising path toward building
  robust and generalizable multimodal agents.
paper_ai_summary: Visual Agentic Reinforcement Fine-Tuning enhances Large Vision-Language Models for flexible image manipulation and web-based reasoning, outperforming existing models on multi-modal benchmarks.
paper_ai_keywords: Visual Agentic Reinforcement Fine-Tuning Large Vision-Language Models Multi-modal Agentic Tool Bench MAT-Search MAT-Coding multi-hop QA benchmarks