title: Seg-Zero: Reasoning-Chain Guided Segmentation via Cognitive
    Reinforcement
summary: Traditional methods for reasoning segmentation rely on supervised fine-tuning
  with categorical labels and simple descriptions, limiting its out-of-domain
  generalization and lacking explicit reasoning processes. To address these
  limitations, we propose Seg-Zero, a novel framework that demonstrates
  remarkable generalizability and derives explicit chain-of-thought reasoning
  through cognitive reinforcement. Seg-Zero introduces a decoupled architecture
  consisting of a reasoning model and a segmentation model. The reasoning model
  interprets user intentions, generates explicit reasoning chains, and produces
  positional prompts, which are subsequently used by the segmentation model to
  generate precious pixel-level masks. We design a sophisticated reward mechanism
  that integrates both format and accuracy rewards to effectively guide
  optimization directions. Trained exclusively via reinforcement learning with
  GRPO and without explicit reasoning data, Seg-Zero achieves robust zero-shot
  generalization and exhibits emergent test-time reasoning capabilities.
  Experiments show that Seg-Zero-7B achieves a zero-shot performance of 57.5 on
  the ReasonSeg benchmark, surpassing the prior LISA-7B by 18\%. This significant
  improvement highlights Seg-Zero's ability to generalize across domains while
  presenting an explicit reasoning process. Code is available at
  https://github.com/dvlab-research/Seg-Zero.
paper_authors._id: 67cf990ca80a73999cc816c3
paper_authors.name: Yuqi Liu
paper_authors._id: 67cf990ca80a73999cc816c4
paper_authors.name: Bohao Peng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-11T14:51:05.435Z
paper_authors._id: 673a10f911b7efeeedabc252
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/T7ySn7F0pTVCvRdcvMz3d.png
paper_authors.fullname: Bohao Peng
paper_authors.type: user
paper_authors.user: BoHao0326
paper_authors._id: 67cf990ca80a73999cc816c5
paper_authors.name: Zhisheng Zhong
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-11T14:50:58.642Z
paper_authors._id: 65d882d30f35ed3f52d3ae2c
paper_authors.avatarUrl: /avatars/22cda67c3fcd7150320ec3551eda90f5.svg
paper_authors.fullname: Zhisheng Zhong
paper_authors.type: user
paper_authors.user: zszhong
paper_authors._id: 67cf990ca80a73999cc816c6
paper_authors.name: Zihao Yue
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-13T08:13:34.470Z
paper_authors._id: 6517f0df593b3af3120d242e
paper_authors.avatarUrl: /avatars/bc28ceb48b206502d8cf9e1c2e130066.svg
paper_authors.fullname: Zihao Yue
paper_authors.type: user
paper_authors.user: yuezih
paper_authors._id: 67cf990ca80a73999cc816c7
paper_authors.name: Fanbin Lu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-11T14:51:21.227Z
paper_authors._id: 645b6094bc7518912e1fbc34
paper_authors.avatarUrl: /avatars/5e1a875ba3ce350e71fe7049ca6a44c1.svg
paper_authors.fullname: Fanbin Lu
paper_authors.type: user
paper_authors.user: Fanbin
paper_authors._id: 67cf990ca80a73999cc816c8
paper_authors.name: Bei Yu
paper_authors._id: 67cf990ca80a73999cc816c9
paper_authors.name: Jiaya Jia
paper_title: Seg-Zero: Reasoning-Chain Guided Segmentation via Cognitive
    Reinforcement
paper_summary: Traditional methods for reasoning segmentation rely on supervised fine-tuning
  with categorical labels and simple descriptions, limiting its out-of-domain
  generalization and lacking explicit reasoning processes. To address these
  limitations, we propose Seg-Zero, a novel framework that demonstrates
  remarkable generalizability and derives explicit chain-of-thought reasoning
  through cognitive reinforcement. Seg-Zero introduces a decoupled architecture
  consisting of a reasoning model and a segmentation model. The reasoning model
  interprets user intentions, generates explicit reasoning chains, and produces
  positional prompts, which are subsequently used by the segmentation model to
  generate precious pixel-level masks. We design a sophisticated reward mechanism
  that integrates both format and accuracy rewards to effectively guide
  optimization directions. Trained exclusively via reinforcement learning with
  GRPO and without explicit reasoning data, Seg-Zero achieves robust zero-shot
  generalization and exhibits emergent test-time reasoning capabilities.
  Experiments show that Seg-Zero-7B achieves a zero-shot performance of 57.5 on
  the ReasonSeg benchmark, surpassing the prior LISA-7B by 18\%. This significant
  improvement highlights Seg-Zero's ability to generalize across domains while
  presenting an explicit reasoning process. Code is available at
  https://github.com/dvlab-research/Seg-Zero.
paper_ai_summary: Seg-Zero, a novel framework using cognitive reinforcement, achieves robust zero-shot generalization in segmentation with explicit reasoning capabilities.
paper_ai_keywords: reasoning segmentation supervised fine-tuning categorical labels explicit reasoning processes decoupled architecture reasoning model segmentation model positional prompts cognitive reinforcement reward mechanism reinforcement learning GRPO zero-shot performance ReasonSeg benchmark