title: One RL to See Them All: Visual Triple Unified Reinforcement Learning
summary: Reinforcement learning (RL) has significantly advanced the reasoning
  capabilities of vision-language models (VLMs). However, the use of RL beyond
  reasoning tasks remains largely unexplored, especially for perceptionintensive
  tasks like object detection and grounding. We propose V-Triune, a Visual Triple
  Unified Reinforcement Learning system that enables VLMs to jointly learn visual
  reasoning and perception tasks within a single training pipeline. V-Triune
  comprises triple complementary components: Sample-Level Data Formatting (to
  unify diverse task inputs), Verifier-Level Reward Computation (to deliver
  custom rewards via specialized verifiers) , and Source-Level Metric Monitoring
  (to diagnose problems at the data-source level). We further introduce a novel
  Dynamic IoU reward, which provides adaptive, progressive, and definite feedback
  for perception tasks handled by V-Triune. Our approach is instantiated within
  off-the-shelf RL training framework using open-source 7B and 32B backbone
  models. The resulting model, dubbed Orsta (One RL to See Them All),
  demonstrates consistent improvements across both reasoning and perception
  tasks. This broad capability is significantly shaped by its training on a
  diverse dataset, constructed around four representative visual reasoning tasks
  (Math, Puzzle, Chart, and Science) and four visual perception tasks (Grounding,
  Detection, Counting, and OCR). Subsequently, Orsta achieves substantial gains
  on MEGA-Bench Core, with improvements ranging from +2.1 to an impressive +14.1
  across its various 7B and 32B model variants, with performance benefits
  extending to a wide range of downstream tasks. These results highlight the
  effectiveness and scalability of our unified RL approach for VLMs. The V-Triune
  system, along with the Orsta models, is publicly available at
  https://github.com/MiniMax-AI.
paper_authors._id: 6833cf89df7cbb5c087a4caa
paper_authors.name: Yan Ma
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-30T06:55:09.156Z
paper_authors._id: 633fc70529b5a95f6e15a6b7
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/633fc70529b5a95f6e15a6b7/Fzh7wWuqU-fBbzdupOUtF.jpeg
paper_authors.fullname: Yan Ma
paper_authors.type: user
paper_authors.user: ManTle
paper_authors._id: 6833cf89df7cbb5c087a4cab
paper_authors.name: Linge Du
paper_authors._id: 6833cf89df7cbb5c087a4cac
paper_authors.name: Xuyang Shen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-26T08:10:03.920Z
paper_authors._id: 642e4d4d6748dd4f8eeb7732
paper_authors.avatarUrl: /avatars/fd911e9143d1a7aedd21a7d611543fcc.svg
paper_authors.fullname: Xuyang Shen
paper_authors.type: user
paper_authors.user: Ryan1122
paper_authors._id: 6833cf89df7cbb5c087a4cad
paper_authors.name: Shaoxiang Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-26T14:27:11.208Z
paper_authors._id: 6829f5feb461b85f5dd0f036
paper_authors.avatarUrl: /avatars/a1308c337ee366ad06819cbd81dbdcb3.svg
paper_authors.fullname: shaoxiang
paper_authors.type: user
paper_authors.user: chenshaoxiang1
paper_authors._id: 6833cf89df7cbb5c087a4cae
paper_authors.name: Pengfei Li
paper_authors._id: 6833cf89df7cbb5c087a4caf
paper_authors.name: Qibing Ren
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-26T14:26:48.219Z
paper_authors._id: 642cede11f576acdab6520e8
paper_authors.avatarUrl: /avatars/95c57579beea2bfe703380a370dc2322.svg
paper_authors.fullname: renqibing
paper_authors.type: user
paper_authors.user: renqibing
paper_authors._id: 6833cf89df7cbb5c087a4cb0
paper_authors.name: Lizhuang Ma
paper_authors._id: 6833cf89df7cbb5c087a4cb1
paper_authors.name: Yuchao Dai
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-26T14:26:30.646Z
paper_authors._id: 647df7670ed7d0c8760b00ed
paper_authors.avatarUrl: /avatars/180dfa736af2aac64cb459ee4563dc61.svg
paper_authors.fullname: Yuchao Dai
paper_authors.type: user
paper_authors.user: daiyuchao
paper_authors._id: 6833cf89df7cbb5c087a4cb2
paper_authors.name: Pengfei Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-26T14:26:36.896Z
paper_authors._id: 6144a0c4ff1146bbd84d9865
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1661715958139-6144a0c4ff1146bbd84d9865.png
paper_authors.fullname: Pengfei Liu
paper_authors.type: user
paper_authors.user: Pengfei
paper_authors._id: 6833cf89df7cbb5c087a4cb3
paper_authors.name: Junjie Yan
paper_title: One RL to See Them All: Visual Triple Unified Reinforcement Learning
paper_summary: Reinforcement learning (RL) has significantly advanced the reasoning
  capabilities of vision-language models (VLMs). However, the use of RL beyond
  reasoning tasks remains largely unexplored, especially for perceptionintensive
  tasks like object detection and grounding. We propose V-Triune, a Visual Triple
  Unified Reinforcement Learning system that enables VLMs to jointly learn visual
  reasoning and perception tasks within a single training pipeline. V-Triune
  comprises triple complementary components: Sample-Level Data Formatting (to
  unify diverse task inputs), Verifier-Level Reward Computation (to deliver
  custom rewards via specialized verifiers) , and Source-Level Metric Monitoring
  (to diagnose problems at the data-source level). We further introduce a novel
  Dynamic IoU reward, which provides adaptive, progressive, and definite feedback
  for perception tasks handled by V-Triune. Our approach is instantiated within
  off-the-shelf RL training framework using open-source 7B and 32B backbone
  models. The resulting model, dubbed Orsta (One RL to See Them All),
  demonstrates consistent improvements across both reasoning and perception
  tasks. This broad capability is significantly shaped by its training on a
  diverse dataset, constructed around four representative visual reasoning tasks
  (Math, Puzzle, Chart, and Science) and four visual perception tasks (Grounding,
  Detection, Counting, and OCR). Subsequently, Orsta achieves substantial gains
  on MEGA-Bench Core, with improvements ranging from +2.1 to an impressive +14.1
  across its various 7B and 32B model variants, with performance benefits
  extending to a wide range of downstream tasks. These results highlight the
  effectiveness and scalability of our unified RL approach for VLMs. The V-Triune
  system, along with the Orsta models, is publicly available at
  https://github.com/MiniMax-AI.
paper_ai_summary: A unified reinforcement learning system, V-Triune, combines visual reasoning and perception tasks in vision-language models through a single training pipeline, achieving significant improvements across various tasks.
paper_ai_keywords: visual triple unified reinforcement learning sample-level data formatting verifier-level reward computation source-level metric monitoring dynamic IoU reward reinforcement learning vision-language models object detection grounding Orsta MEGA-Bench Core