PixelThink: Towards Efficient Chain-of-Pixel Reasoning
Existing reasoning segmentation approaches typically fine-tune multimodal
large language models (MLLMs) using image-text pairs and corresponding mask
labels. However, they exhibit limited generalization to out-of-distribution
scenarios without an explicit reasoning process. Although recent efforts
leverage reinforcement learning through group-relative policy optimization
(GRPO) to enhance reasoning ability, they often suffer from overthinking -
producing uniformly verbose reasoning chains irrespective of task complexity.
This results in elevated computational costs and limited control over reasoning
quality. To address this problem, we propose PixelThink, a simple yet effective
scheme that integrates externally estimated task difficulty and internally
measured model uncertainty to regulate reasoning generation within a
reinforcement learning paradigm. The model learns to compress reasoning length
in accordance with scene complexity and predictive confidence. To support
comprehensive evaluation, we introduce ReasonSeg-Diff, an extended benchmark
with annotated reasoning references and difficulty scores, along with a suite
of metrics designed to assess segmentation accuracy, reasoning quality, and
efficiency jointly. Experimental results demonstrate that the proposed approach
improves both reasoning efficiency and overall segmentation performance. Our
work contributes novel perspectives towards efficient and interpretable
multimodal understanding. The code and model will be publicly available.
6839080fd73e6015a1acc507
Song Wang
claimed_verified
2025-05-30T06:54:25.990Z
66863d26e2b71e3d09189ae9
/avatars/3c0e6f30e053f2e622ae75e1dc43edba.svg
Song Wang
user
songw-zju
6839080fd73e6015a1acc508
Gongfan Fang
6839080fd73e6015a1acc509
Lingdong Kong
6839080fd73e6015a1acc50a
Xiangtai Li
claimed_verified
2025-06-11T08:37:13.361Z
63958b4414513eaf9029ebf1
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/U1g5H071pWRswGAG9UTpo.png
Xiangtai Li
user
LXT
6839080fd73e6015a1acc50b
Jianyun Xu
6839080fd73e6015a1acc50c
Sheng Yang
6839080fd73e6015a1acc50d
Qiang Li
6839080fd73e6015a1acc50e
Jianke Zhu
6839080fd73e6015a1acc50f
Xinchao Wang
PixelThink: Towards Efficient Chain-of-Pixel Reasoning
Existing reasoning segmentation approaches typically fine-tune multimodal
large language models (MLLMs) using image-text pairs and corresponding mask
labels. However, they exhibit limited generalization to out-of-distribution
scenarios without an explicit reasoning process. Although recent efforts
leverage reinforcement learning through group-relative policy optimization
(GRPO) to enhance reasoning ability, they often suffer from overthinking -
producing uniformly verbose reasoning chains irrespective of task complexity.
This results in elevated computational costs and limited control over reasoning
quality. To address this problem, we propose PixelThink, a simple yet effective
scheme that integrates externally estimated task difficulty and internally
measured model uncertainty to regulate reasoning generation within a
reinforcement learning paradigm. The model learns to compress reasoning length
in accordance with scene complexity and predictive confidence. To support
comprehensive evaluation, we introduce ReasonSeg-Diff, an extended benchmark
with annotated reasoning references and difficulty scores, along with a suite
of metrics designed to assess segmentation accuracy, reasoning quality, and
efficiency jointly. Experimental results demonstrate that the proposed approach
improves both reasoning efficiency and overall segmentation performance. Our
work contributes novel perspectives towards efficient and interpretable
multimodal understanding. The code and model will be publicly available.
PixelThink enhances reasoning segmentation by integrating task difficulty and model uncertainty in reinforcement learning, improving efficiency and accuracy.
multimodal large language models image-text pairs reinforcement learning group-relative policy optimization reasoning chains PixelThink task difficulty model uncertainty reasoning generation scene complexity predictive confidence reasoning quality ReasonSeg-Diff segmentation accuracy efficiency