SeC: Advancing Complex Video Object Segmentation via Progressive Concept
  Construction
Video Object Segmentation (VOS) is a core task in computer vision, requiring
models to track and segment target objects across video frames. Despite notable
advances with recent efforts, current techniques still lag behind human
capabilities in handling drastic visual variations, occlusions, and complex
scene changes. This limitation arises from their reliance on appearance
matching, neglecting the human-like conceptual understanding of objects that
enables robust identification across temporal dynamics. Motivated by this gap,
we propose Segment Concept (SeC), a concept-driven segmentation framework that
shifts from conventional feature matching to the progressive construction and
utilization of high-level, object-centric representations. SeC employs Large
Vision-Language Models (LVLMs) to integrate visual cues across diverse frames,
constructing robust conceptual priors. During inference, SeC forms a
comprehensive semantic representation of the target based on processed frames,
realizing robust segmentation of follow-up frames. Furthermore, SeC adaptively
balances LVLM-based semantic reasoning with enhanced feature matching,
dynamically adjusting computational efforts based on scene complexity. To
rigorously assess VOS methods in scenarios demanding high-level conceptual
reasoning and robust semantic understanding, we introduce the Semantic Complex
Scenarios Video Object Segmentation benchmark (SeCVOS). SeCVOS comprises 160
manually annotated multi-scenario videos designed to challenge models with
substantial appearance variations and dynamic scene transformations. In
particular, SeC achieves an 11.8-point improvement over SAM 2.1 on SeCVOS,
establishing a new state-of-the-art in concept-aware video object segmentation.
687efbab33947f780d9b4aee
Zhixiong Zhang
claimed_verified
2025-07-22T07:48:03.855Z
65ab5332043d53781a115475
https://cdn-avatars.huggingface.co/v1/production/uploads/65ab5332043d53781a115475/UaxSFDWteYsByzx7G_KKy.jpeg
Zhixiong Zhang
user
rookiexiong
687efbab33947f780d9b4aef
Shuangrui Ding
687efbab33947f780d9b4af0
Xiaoyi Dong
687efbab33947f780d9b4af1
Songxin He
687efbab33947f780d9b4af2
Jianfan Lin
687efbab33947f780d9b4af3
Junsong Tang
687efbab33947f780d9b4af4
Yuhang Zang
claimed_verified
2025-07-22T13:05:45.709Z
63859cf3b2906edaf83af9f0
https://cdn-avatars.huggingface.co/v1/production/uploads/63859cf3b2906edaf83af9f0/mCgynEtoXdILvzyoPexii.png
Yuhang Zang
user
yuhangzang
687efbab33947f780d9b4af5
Yuhang Cao
687efbab33947f780d9b4af6
Dahua Lin
687efbab33947f780d9b4af7
Jiaqi Wang
SeC: Advancing Complex Video Object Segmentation via Progressive Concept
  Construction
Video Object Segmentation (VOS) is a core task in computer vision, requiring
models to track and segment target objects across video frames. Despite notable
advances with recent efforts, current techniques still lag behind human
capabilities in handling drastic visual variations, occlusions, and complex
scene changes. This limitation arises from their reliance on appearance
matching, neglecting the human-like conceptual understanding of objects that
enables robust identification across temporal dynamics. Motivated by this gap,
we propose Segment Concept (SeC), a concept-driven segmentation framework that
shifts from conventional feature matching to the progressive construction and
utilization of high-level, object-centric representations. SeC employs Large
Vision-Language Models (LVLMs) to integrate visual cues across diverse frames,
constructing robust conceptual priors. During inference, SeC forms a
comprehensive semantic representation of the target based on processed frames,
realizing robust segmentation of follow-up frames. Furthermore, SeC adaptively
balances LVLM-based semantic reasoning with enhanced feature matching,
dynamically adjusting computational efforts based on scene complexity. To
rigorously assess VOS methods in scenarios demanding high-level conceptual
reasoning and robust semantic understanding, we introduce the Semantic Complex
Scenarios Video Object Segmentation benchmark (SeCVOS). SeCVOS comprises 160
manually annotated multi-scenario videos designed to challenge models with
substantial appearance variations and dynamic scene transformations. In
particular, SeC achieves an 11.8-point improvement over SAM 2.1 on SeCVOS,
establishing a new state-of-the-art in concept-aware video object segmentation.
A concept-driven segmentation framework using Large Vision-Language Models improves video object segmentation by integrating high-level semantic reasoning and adapting to scene complexity.
Video Object Segmentation VOS Large Vision-Language Models LVLMs semantic representation feature matching SeCVOS concept-aware segmentation