title: Active-O3: Empowering Multimodal Large Language Models with Active
    Perception via GRPO
summary: Active vision, also known as active perception, refers to the process of
  actively selecting where and how to look in order to gather task-relevant
  information. It is a critical component of efficient perception and
  decision-making in humans and advanced embodied agents. Recently, the use of
  Multimodal Large Language Models (MLLMs) as central planning and
  decision-making modules in robotic systems has gained extensive attention.
  However, despite the importance of active perception in embodied intelligence,
  there is little to no exploration of how MLLMs can be equipped with or learn
  active perception capabilities. In this paper, we first provide a systematic
  definition of MLLM-based active perception tasks. We point out that the
  recently proposed GPT-o3 model's zoom-in search strategy can be regarded as a
  special case of active perception; however, it still suffers from low search
  efficiency and inaccurate region selection. To address these issues, we propose
  ACTIVE-O3, a purely reinforcement learning based training framework built on
  top of GRPO, designed to equip MLLMs with active perception capabilities. We
  further establish a comprehensive benchmark suite to evaluate ACTIVE-O3 across
  both general open-world tasks, such as small-object and dense object grounding,
  and domain-specific scenarios, including small object detection in remote
  sensing and autonomous driving, as well as fine-grained interactive
  segmentation. In addition, ACTIVE-O3 also demonstrates strong zero-shot
  reasoning abilities on the V* Benchmark, without relying on any explicit
  reasoning data. We hope that our work can provide a simple codebase and
  evaluation protocol to facilitate future research on active perception in
  MLLMs.
paper_authors._id: 6836711b80ed824b28f7a78a
paper_authors.name: Muzhi Zhu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-28T08:57:55.082Z
paper_authors._id: 632179745fc60c44fd91fc33
paper_authors.avatarUrl: /avatars/37d4fefbcc19f091dccffefec9706de2.svg
paper_authors.fullname: zhumuzhi
paper_authors.type: user
paper_authors.user: Z-MU-Z
paper_authors._id: 6836711b80ed824b28f7a78b
paper_authors.name: Hao Zhong
paper_authors._id: 6836711b80ed824b28f7a78c
paper_authors.name: Canyu Zhao
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-28T08:57:48.936Z
paper_authors._id: 646efd223dd912a539e0bd46
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/EOFAv5xvOgJOzuDgh4nSb.png
paper_authors.fullname: Canyu Zhao
paper_authors.type: user
paper_authors.user: Canyu
paper_authors._id: 6836711b80ed824b28f7a78d
paper_authors.name: Zongze Du
paper_authors._id: 6836711b80ed824b28f7a78e
paper_authors.name: Zheng Huang
paper_authors._id: 6836711b80ed824b28f7a78f
paper_authors.name: Mingyu Liu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-28T08:57:51.293Z
paper_authors._id: 652e25d2e647b0ee0a024f26
paper_authors.avatarUrl: /avatars/b5c65cf6c8d0ddc9b8ef0226e0295d56.svg
paper_authors.fullname: Mingyu Liu
paper_authors.type: user
paper_authors.user: MingyuLiu
paper_authors._id: 6836711b80ed824b28f7a790
paper_authors.name: Hao Chen
paper_authors._id: 6836711b80ed824b28f7a791
paper_authors.name: Cheng Zou
paper_authors._id: 6836711b80ed824b28f7a792
paper_authors.name: Jingdong Chen
paper_authors._id: 6836711b80ed824b28f7a793
paper_authors.name: Ming Yang
paper_authors._id: 6836711b80ed824b28f7a794
paper_authors.name: Chunhua Shen
paper_title: Active-O3: Empowering Multimodal Large Language Models with Active
    Perception via GRPO
paper_summary: Active vision, also known as active perception, refers to the process of
  actively selecting where and how to look in order to gather task-relevant
  information. It is a critical component of efficient perception and
  decision-making in humans and advanced embodied agents. Recently, the use of
  Multimodal Large Language Models (MLLMs) as central planning and
  decision-making modules in robotic systems has gained extensive attention.
  However, despite the importance of active perception in embodied intelligence,
  there is little to no exploration of how MLLMs can be equipped with or learn
  active perception capabilities. In this paper, we first provide a systematic
  definition of MLLM-based active perception tasks. We point out that the
  recently proposed GPT-o3 model's zoom-in search strategy can be regarded as a
  special case of active perception; however, it still suffers from low search
  efficiency and inaccurate region selection. To address these issues, we propose
  ACTIVE-O3, a purely reinforcement learning based training framework built on
  top of GRPO, designed to equip MLLMs with active perception capabilities. We
  further establish a comprehensive benchmark suite to evaluate ACTIVE-O3 across
  both general open-world tasks, such as small-object and dense object grounding,
  and domain-specific scenarios, including small object detection in remote
  sensing and autonomous driving, as well as fine-grained interactive
  segmentation. In addition, ACTIVE-O3 also demonstrates strong zero-shot
  reasoning abilities on the V* Benchmark, without relying on any explicit
  reasoning data. We hope that our work can provide a simple codebase and
  evaluation protocol to facilitate future research on active perception in
  MLLMs.
paper_ai_summary: A reinforcement learning framework, ACTIVE-O3, is proposed to equip Multimodal Large Language Models with active perception capabilities and tested across various tasks and benchmarks.
paper_ai_keywords: Multimodal Large Language Models MLLMs active perception GPT-o3 zoom-in search ACTIVE-O3 reinforcement learning GRPO small-object grounding dense object grounding small object detection remote sensing autonomous driving fine-grained interactive segmentation V* Benchmark zero-shot reasoning