title: Omni-R1: Reinforcement Learning for Omnimodal Reasoning via Two-System
    Collaboration
summary: Long-horizon video-audio reasoning and fine-grained pixel understanding
  impose conflicting requirements on omnimodal models: dense temporal coverage
  demands many low-resolution frames, whereas precise grounding calls for
  high-resolution inputs. We tackle this trade-off with a two-system
  architecture: a Global Reasoning System selects informative keyframes and
  rewrites the task at low spatial cost, while a Detail Understanding System
  performs pixel-level grounding on the selected high-resolution snippets.
  Because ``optimal'' keyframe selection and reformulation are ambiguous and hard
  to supervise, we formulate them as a reinforcement learning (RL) problem and
  present Omni-R1, an end-to-end RL framework built on Group Relative Policy
  Optimization. Omni-R1 trains the Global Reasoning System through hierarchical
  rewards obtained via online collaboration with the Detail Understanding System,
  requiring only one epoch of RL on small task splits.
    Experiments on two challenging benchmarks, namely Referring Audio-Visual
  Segmentation (RefAVS) and Reasoning Video Object Segmentation (REVOS), show
  that Omni-R1 not only surpasses strong supervised baselines but also
  outperforms specialized state-of-the-art models, while substantially improving
  out-of-domain generalization and mitigating multimodal hallucination. Our
  results demonstrate the first successful application of RL to large-scale
  omnimodal reasoning and highlight a scalable path toward universally foundation
  models.
paper_authors._id: 68352e44c829f2ea1e0484b5
paper_authors.name: Hao Zhong
paper_authors._id: 68352e44c829f2ea1e0484b6
paper_authors.name: Muzhi Zhu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-28T09:00:37.809Z
paper_authors._id: 632179745fc60c44fd91fc33
paper_authors.avatarUrl: /avatars/37d4fefbcc19f091dccffefec9706de2.svg
paper_authors.fullname: zhumuzhi
paper_authors.type: user
paper_authors.user: Z-MU-Z
paper_authors._id: 68352e44c829f2ea1e0484b7
paper_authors.name: Zongze Du
paper_authors._id: 68352e44c829f2ea1e0484b8
paper_authors.name: Zheng Huang
paper_authors._id: 68352e44c829f2ea1e0484b9
paper_authors.name: Canyu Zhao
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-27T07:50:50.579Z
paper_authors._id: 646efd223dd912a539e0bd46
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/EOFAv5xvOgJOzuDgh4nSb.png
paper_authors.fullname: Canyu Zhao
paper_authors.type: user
paper_authors.user: Canyu
paper_authors._id: 68352e44c829f2ea1e0484ba
paper_authors.name: Mingyu Liu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-28T09:00:35.196Z
paper_authors._id: 652e25d2e647b0ee0a024f26
paper_authors.avatarUrl: /avatars/b5c65cf6c8d0ddc9b8ef0226e0295d56.svg
paper_authors.fullname: Mingyu Liu
paper_authors.type: user
paper_authors.user: MingyuLiu
paper_authors._id: 68352e44c829f2ea1e0484bb
paper_authors.name: Wen Wang
paper_authors._id: 68352e44c829f2ea1e0484bc
paper_authors.name: Hao Chen
paper_authors._id: 68352e44c829f2ea1e0484bd
paper_authors.name: Chunhua Shen
paper_title: Omni-R1: Reinforcement Learning for Omnimodal Reasoning via Two-System
    Collaboration
paper_summary: Long-horizon video-audio reasoning and fine-grained pixel understanding
  impose conflicting requirements on omnimodal models: dense temporal coverage
  demands many low-resolution frames, whereas precise grounding calls for
  high-resolution inputs. We tackle this trade-off with a two-system
  architecture: a Global Reasoning System selects informative keyframes and
  rewrites the task at low spatial cost, while a Detail Understanding System
  performs pixel-level grounding on the selected high-resolution snippets.
  Because ``optimal'' keyframe selection and reformulation are ambiguous and hard
  to supervise, we formulate them as a reinforcement learning (RL) problem and
  present Omni-R1, an end-to-end RL framework built on Group Relative Policy
  Optimization. Omni-R1 trains the Global Reasoning System through hierarchical
  rewards obtained via online collaboration with the Detail Understanding System,
  requiring only one epoch of RL on small task splits.
    Experiments on two challenging benchmarks, namely Referring Audio-Visual
  Segmentation (RefAVS) and Reasoning Video Object Segmentation (REVOS), show
  that Omni-R1 not only surpasses strong supervised baselines but also
  outperforms specialized state-of-the-art models, while substantially improving
  out-of-domain generalization and mitigating multimodal hallucination. Our
  results demonstrate the first successful application of RL to large-scale
  omnimodal reasoning and highlight a scalable path toward universally foundation
  models.
paper_ai_summary: An end-to-end reinforcement learning framework, Omni-R1, achieves superior performance in long-horizon video-audio reasoning and fine-grained pixel understanding tasks by combining global reasoning and detail understanding systems.
paper_ai_keywords: reinforcement learning Group Relative Policy Optimization hierarchical rewards Referring Audio-Visual Segmentation Reasoning Video Object Segmentation out-of-domain generalization multimodal hallucination universally foundation models