title: Visual-RFT: Visual Reinforcement Fine-Tuning
summary: Reinforcement Fine-Tuning (RFT) in Large Reasoning Models like OpenAI o1
  learns from feedback on its answers, which is especially useful in applications
  when fine-tuning data is scarce. Recent open-source work like DeepSeek-R1
  demonstrates that reinforcement learning with verifiable reward is one key
  direction in reproducing o1. While the R1-style model has demonstrated success
  in language models, its application in multi-modal domains remains
  under-explored. This work introduces Visual Reinforcement Fine-Tuning
  (Visual-RFT), which further extends the application areas of RFT on visual
  tasks. Specifically, Visual-RFT first uses Large Vision-Language Models (LVLMs)
  to generate multiple responses containing reasoning tokens and final answers
  for each input, and then uses our proposed visual perception verifiable reward
  functions to update the model via the policy optimization algorithm such as
  Group Relative Policy Optimization (GRPO). We design different verifiable
  reward functions for different perception tasks, such as the Intersection over
  Union (IoU) reward for object detection. Experimental results on fine-grained
  image classification, few-shot object detection, reasoning grounding, as well
  as open-vocabulary object detection benchmarks show the competitive performance
  and advanced generalization ability of Visual-RFT compared with Supervised
  Fine-tuning (SFT). For example, Visual-RFT improves accuracy by 24.3% over
  the baseline in one-shot fine-grained image classification with around 100
  samples. In few-shot object detection, Visual-RFT also exceeds the baseline by
  21.9 on COCO's two-shot setting and 15.4 on LVIS. Our Visual-RFT represents
  a paradigm shift in fine-tuning LVLMs, offering a data-efficient, reward-driven
  approach that enhances reasoning and adaptability for domain-specific tasks.
paper_authors._id: 67c6816614a1bf9855188b8b
paper_authors.name: Ziyu Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-04T09:12:57.481Z
paper_authors._id: 66fe1334ff3ee1f7569fab6d
paper_authors.avatarUrl: /avatars/6868b1a545028a9b8bbded52490dc093.svg
paper_authors.fullname: ziyuliu
paper_authors.type: user
paper_authors.user: ziyuliu
paper_authors._id: 67c6816614a1bf9855188b8c
paper_authors.name: Zeyi Sun
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-04T08:35:03.275Z
paper_authors._id: 63fda3fced9eead590ff6918
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1677566802735-noauth.jpeg
paper_authors.fullname: Zeyi Sun
paper_authors.type: user
paper_authors.user: Zery
paper_authors._id: 67c6816614a1bf9855188b8d
paper_authors.name: Yuhang Zang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-04T09:12:32.723Z
paper_authors._id: 63859cf3b2906edaf83af9f0
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63859cf3b2906edaf83af9f0/mCgynEtoXdILvzyoPexii.png
paper_authors.fullname: Yuhang Zang
paper_authors.type: user
paper_authors.user: yuhangzang
paper_authors._id: 67c6816614a1bf9855188b8e
paper_authors.name: Xiaoyi Dong
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-04T09:12:25.627Z
paper_authors._id: 67c0849ee08c178ef8d4e05c
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/mQ6VdnjZnRhb0H_waPclo.png
paper_authors.fullname: Xiaoyi Dong
paper_authors.type: user
paper_authors.user: sweetFruit
paper_authors._id: 67c6816614a1bf9855188b8f
paper_authors.name: Yuhang Cao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-04T09:12:19.177Z
paper_authors._id: 65000bef18830fabea469fdd
paper_authors.avatarUrl: /avatars/b320c77dfad039d9f9c54127f610d44f.svg
paper_authors.fullname: Cao Yuhang
paper_authors.type: user
paper_authors.user: yhcao
paper_authors._id: 67c6816614a1bf9855188b90
paper_authors.name: Haodong Duan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-04T09:12:05.281Z
paper_authors._id: 63ee1379190ddd6214efd73a
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1676546883247-noauth.png
paper_authors.fullname: HAODONG DUAN
paper_authors.type: user
paper_authors.user: KennyUTC
paper_authors._id: 67c6816614a1bf9855188b91
paper_authors.name: Dahua Lin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-04T09:11:57.087Z
paper_authors._id: 636317ed80c1a705a6eff396
paper_authors.avatarUrl: /avatars/3db090e101b916d9256d0d3e043db71d.svg
paper_authors.fullname: Dahua Lin
paper_authors.type: user
paper_authors.user: lindahua
paper_authors._id: 67c6816614a1bf9855188b92
paper_authors.name: Jiaqi Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-04T09:11:48.889Z
paper_authors._id: 64638c4d51fa6e63060521b5
paper_authors.avatarUrl: /avatars/c863ace5b1dc788a341bcf4ddbdfaec1.svg
paper_authors.fullname: JIaqi
paper_authors.type: user
paper_authors.user: Jiaqiwang
paper_title: Visual-RFT: Visual Reinforcement Fine-Tuning
paper_summary: Reinforcement Fine-Tuning (RFT) in Large Reasoning Models like OpenAI o1
  learns from feedback on its answers, which is especially useful in applications
  when fine-tuning data is scarce. Recent open-source work like DeepSeek-R1
  demonstrates that reinforcement learning with verifiable reward is one key
  direction in reproducing o1. While the R1-style model has demonstrated success
  in language models, its application in multi-modal domains remains
  under-explored. This work introduces Visual Reinforcement Fine-Tuning
  (Visual-RFT), which further extends the application areas of RFT on visual
  tasks. Specifically, Visual-RFT first uses Large Vision-Language Models (LVLMs)
  to generate multiple responses containing reasoning tokens and final answers
  for each input, and then uses our proposed visual perception verifiable reward
  functions to update the model via the policy optimization algorithm such as
  Group Relative Policy Optimization (GRPO). We design different verifiable
  reward functions for different perception tasks, such as the Intersection over
  Union (IoU) reward for object detection. Experimental results on fine-grained
  image classification, few-shot object detection, reasoning grounding, as well
  as open-vocabulary object detection benchmarks show the competitive performance
  and advanced generalization ability of Visual-RFT compared with Supervised
  Fine-tuning (SFT). For example, Visual-RFT improves accuracy by 24.3% over
  the baseline in one-shot fine-grained image classification with around 100
  samples. In few-shot object detection, Visual-RFT also exceeds the baseline by
  21.9 on COCO's two-shot setting and 15.4 on LVIS. Our Visual-RFT represents
  a paradigm shift in fine-tuning LVLMs, offering a data-efficient, reward-driven
  approach that enhances reasoning and adaptability for domain-specific tasks.
paper_ai_summary: Visual Reinforcement Fine-Tuning (Visual-RFT) enhances Large Vision-Language Models (LVLMs) through reinforcement learning with visual perception verifiable rewards, achieving competitive performance in various visual and reasoning tasks with limited data.
paper_ai_keywords: Reinforcement Fine-Tuning (RFT) Visual Reinforcement Fine-Tuning (Visual-RFT) Large Vision-Language Models (LVLMs) reasoning tokens policy optimization algorithm Group Relative Policy Optimization (GRPO) Intersection over Union (IoU) fine-grained image classification few-shot object detection reasoning grounding open-vocabulary object detection Supervised Fine-tuning (SFT)