title: PixelThink: Towards Efficient Chain-of-Pixel Reasoning
summary: Existing reasoning segmentation approaches typically fine-tune multimodal
  large language models (MLLMs) using image-text pairs and corresponding mask
  labels. However, they exhibit limited generalization to out-of-distribution
  scenarios without an explicit reasoning process. Although recent efforts
  leverage reinforcement learning through group-relative policy optimization
  (GRPO) to enhance reasoning ability, they often suffer from overthinking -
  producing uniformly verbose reasoning chains irrespective of task complexity.
  This results in elevated computational costs and limited control over reasoning
  quality. To address this problem, we propose PixelThink, a simple yet effective
  scheme that integrates externally estimated task difficulty and internally
  measured model uncertainty to regulate reasoning generation within a
  reinforcement learning paradigm. The model learns to compress reasoning length
  in accordance with scene complexity and predictive confidence. To support
  comprehensive evaluation, we introduce ReasonSeg-Diff, an extended benchmark
  with annotated reasoning references and difficulty scores, along with a suite
  of metrics designed to assess segmentation accuracy, reasoning quality, and
  efficiency jointly. Experimental results demonstrate that the proposed approach
  improves both reasoning efficiency and overall segmentation performance. Our
  work contributes novel perspectives towards efficient and interpretable
  multimodal understanding. The code and model will be publicly available.
paper_authors._id: 6839080fd73e6015a1acc507
paper_authors.name: Song Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-30T06:54:25.990Z
paper_authors._id: 66863d26e2b71e3d09189ae9
paper_authors.avatarUrl: /avatars/3c0e6f30e053f2e622ae75e1dc43edba.svg
paper_authors.fullname: Song Wang
paper_authors.type: user
paper_authors.user: songw-zju
paper_authors._id: 6839080fd73e6015a1acc508
paper_authors.name: Gongfan Fang
paper_authors._id: 6839080fd73e6015a1acc509
paper_authors.name: Lingdong Kong
paper_authors._id: 6839080fd73e6015a1acc50a
paper_authors.name: Xiangtai Li
paper_authors._id: 6839080fd73e6015a1acc50b
paper_authors.name: Jianyun Xu
paper_authors._id: 6839080fd73e6015a1acc50c
paper_authors.name: Sheng Yang
paper_authors._id: 6839080fd73e6015a1acc50d
paper_authors.name: Qiang Li
paper_authors._id: 6839080fd73e6015a1acc50e
paper_authors.name: Jianke Zhu
paper_authors._id: 6839080fd73e6015a1acc50f
paper_authors.name: Xinchao Wang
paper_title: PixelThink: Towards Efficient Chain-of-Pixel Reasoning
paper_summary: Existing reasoning segmentation approaches typically fine-tune multimodal
  large language models (MLLMs) using image-text pairs and corresponding mask
  labels. However, they exhibit limited generalization to out-of-distribution
  scenarios without an explicit reasoning process. Although recent efforts
  leverage reinforcement learning through group-relative policy optimization
  (GRPO) to enhance reasoning ability, they often suffer from overthinking -
  producing uniformly verbose reasoning chains irrespective of task complexity.
  This results in elevated computational costs and limited control over reasoning
  quality. To address this problem, we propose PixelThink, a simple yet effective
  scheme that integrates externally estimated task difficulty and internally
  measured model uncertainty to regulate reasoning generation within a
  reinforcement learning paradigm. The model learns to compress reasoning length
  in accordance with scene complexity and predictive confidence. To support
  comprehensive evaluation, we introduce ReasonSeg-Diff, an extended benchmark
  with annotated reasoning references and difficulty scores, along with a suite
  of metrics designed to assess segmentation accuracy, reasoning quality, and
  efficiency jointly. Experimental results demonstrate that the proposed approach
  improves both reasoning efficiency and overall segmentation performance. Our
  work contributes novel perspectives towards efficient and interpretable
  multimodal understanding. The code and model will be publicly available.
paper_ai_summary: PixelThink enhances reasoning segmentation by integrating task difficulty and model uncertainty in reinforcement learning, improving efficiency and accuracy.
paper_ai_keywords: multimodal large language models image-text pairs reinforcement learning group-relative policy optimization reasoning chains PixelThink task difficulty model uncertainty reasoning generation scene complexity predictive confidence reasoning quality ReasonSeg-Diff segmentation accuracy efficiency