One RL to See Them All: Visual Triple Unified Reinforcement Learning
Reinforcement learning (RL) has significantly advanced the reasoning
capabilities of vision-language models (VLMs). However, the use of RL beyond
reasoning tasks remains largely unexplored, especially for perceptionintensive
tasks like object detection and grounding. We propose V-Triune, a Visual Triple
Unified Reinforcement Learning system that enables VLMs to jointly learn visual
reasoning and perception tasks within a single training pipeline. V-Triune
comprises triple complementary components: Sample-Level Data Formatting (to
unify diverse task inputs), Verifier-Level Reward Computation (to deliver
custom rewards via specialized verifiers) , and Source-Level Metric Monitoring
(to diagnose problems at the data-source level). We further introduce a novel
Dynamic IoU reward, which provides adaptive, progressive, and definite feedback
for perception tasks handled by V-Triune. Our approach is instantiated within
off-the-shelf RL training framework using open-source 7B and 32B backbone
models. The resulting model, dubbed Orsta (One RL to See Them All),
demonstrates consistent improvements across both reasoning and perception
tasks. This broad capability is significantly shaped by its training on a
diverse dataset, constructed around four representative visual reasoning tasks
(Math, Puzzle, Chart, and Science) and four visual perception tasks (Grounding,
Detection, Counting, and OCR). Subsequently, Orsta achieves substantial gains
on MEGA-Bench Core, with improvements ranging from +2.1 to an impressive +14.1
across its various 7B and 32B model variants, with performance benefits
extending to a wide range of downstream tasks. These results highlight the
effectiveness and scalability of our unified RL approach for VLMs. The V-Triune
system, along with the Orsta models, is publicly available at
https://github.com/MiniMax-AI.
6833cf89df7cbb5c087a4caa
Yan Ma
claimed_verified
2025-05-30T06:55:09.156Z
633fc70529b5a95f6e15a6b7
https://cdn-avatars.huggingface.co/v1/production/uploads/633fc70529b5a95f6e15a6b7/Fzh7wWuqU-fBbzdupOUtF.jpeg
Yan Ma
user
ManTle
6833cf89df7cbb5c087a4cab
Linge Du
6833cf89df7cbb5c087a4cac
Xuyang Shen
claimed_verified
2025-05-26T08:10:03.920Z
642e4d4d6748dd4f8eeb7732
/avatars/fd911e9143d1a7aedd21a7d611543fcc.svg
Xuyang Shen
user
Ryan1122
6833cf89df7cbb5c087a4cad
Shaoxiang Chen
admin_assigned
2025-05-26T14:27:11.208Z
6829f5feb461b85f5dd0f036
/avatars/a1308c337ee366ad06819cbd81dbdcb3.svg
shaoxiang
user
chenshaoxiang1
6833cf89df7cbb5c087a4cae
Pengfei Li
6833cf89df7cbb5c087a4caf
Qibing Ren
admin_assigned
2025-05-26T14:26:48.219Z
642cede11f576acdab6520e8
/avatars/95c57579beea2bfe703380a370dc2322.svg
renqibing
user
renqibing
6833cf89df7cbb5c087a4cb0
Lizhuang Ma
6833cf89df7cbb5c087a4cb1
Yuchao Dai
admin_assigned
2025-05-26T14:26:30.646Z
647df7670ed7d0c8760b00ed
/avatars/180dfa736af2aac64cb459ee4563dc61.svg
Yuchao Dai
user
daiyuchao
6833cf89df7cbb5c087a4cb2
Pengfei Liu
admin_assigned
2025-05-26T14:26:36.896Z
6144a0c4ff1146bbd84d9865
https://cdn-avatars.huggingface.co/v1/production/uploads/1661715958139-6144a0c4ff1146bbd84d9865.png
Pengfei Liu
user
Pengfei
6833cf89df7cbb5c087a4cb3
Junjie Yan
One RL to See Them All: Visual Triple Unified Reinforcement Learning
Reinforcement learning (RL) has significantly advanced the reasoning
capabilities of vision-language models (VLMs). However, the use of RL beyond
reasoning tasks remains largely unexplored, especially for perceptionintensive
tasks like object detection and grounding. We propose V-Triune, a Visual Triple
Unified Reinforcement Learning system that enables VLMs to jointly learn visual
reasoning and perception tasks within a single training pipeline. V-Triune
comprises triple complementary components: Sample-Level Data Formatting (to
unify diverse task inputs), Verifier-Level Reward Computation (to deliver
custom rewards via specialized verifiers) , and Source-Level Metric Monitoring
(to diagnose problems at the data-source level). We further introduce a novel
Dynamic IoU reward, which provides adaptive, progressive, and definite feedback
for perception tasks handled by V-Triune. Our approach is instantiated within
off-the-shelf RL training framework using open-source 7B and 32B backbone
models. The resulting model, dubbed Orsta (One RL to See Them All),
demonstrates consistent improvements across both reasoning and perception
tasks. This broad capability is significantly shaped by its training on a
diverse dataset, constructed around four representative visual reasoning tasks
(Math, Puzzle, Chart, and Science) and four visual perception tasks (Grounding,
Detection, Counting, and OCR). Subsequently, Orsta achieves substantial gains
on MEGA-Bench Core, with improvements ranging from +2.1 to an impressive +14.1
across its various 7B and 32B model variants, with performance benefits
extending to a wide range of downstream tasks. These results highlight the
effectiveness and scalability of our unified RL approach for VLMs. The V-Triune
system, along with the Orsta models, is publicly available at
https://github.com/MiniMax-AI.
A unified reinforcement learning system, V-Triune, combines visual reasoning and perception tasks in vision-language models through a single training pipeline, achieving significant improvements across various tasks.
visual triple unified reinforcement learning sample-level data formatting verifier-level reward computation source-level metric monitoring dynamic IoU reward reinforcement learning vision-language models object detection grounding Orsta MEGA-Bench Core