Unsupervised Universal Image Segmentation
Several unsupervised image segmentation approaches have been proposed which
eliminate the need for dense manually-annotated segmentation masks; current
models separately handle either semantic segmentation (e.g., STEGO) or
class-agnostic instance segmentation (e.g., CutLER), but not both (i.e.,
panoptic segmentation). We propose an Unsupervised Universal Segmentation model
(U2Seg) adept at performing various image segmentation tasks -- instance,
semantic and panoptic -- using a novel unified framework. U2Seg generates
pseudo semantic labels for these segmentation tasks via leveraging
self-supervised models followed by clustering; each cluster represents
different semantic and/or instance membership of pixels. We then self-train the
model on these pseudo semantic labels, yielding substantial performance gains
over specialized methods tailored to each task: a +2.6 AP^{box} boost
vs. CutLER in unsupervised instance segmentation on COCO and a +7.0 PixelAcc
increase (vs. STEGO) in unsupervised semantic segmentation on COCOStuff.
Moreover, our method sets up a new baseline for unsupervised panoptic
segmentation, which has not been previously explored. U2Seg is also a strong
pretrained model for few-shot segmentation, surpassing CutLER by +5.0
AP^{mask} when trained on a low-data regime, e.g., only 1% COCO
labels. We hope our simple yet effective method can inspire more research on
unsupervised universal image segmentation.
658e4f96f5a209eeacb4404c
Dantong Niu
658e4f96f5a209eeacb4404d
Xudong Wang
claimed_verified
2024-02-21T08:30:54.754Z
63d6f76f44f1d8fbe58fbd9d
/avatars/184f248a090b543e576a97b7f46ab3ef.svg
Xudong Wang
user
xudongw
658e4f96f5a209eeacb4404e
Xinyang Han
658e4f96f5a209eeacb4404f
Long Lian
658e4f96f5a209eeacb44050
Roei Herzig
claimed_verified
2024-06-27T09:50:01.072Z
667c5764186b27ef806636d3
/avatars/5c08f0109bc0e350624112c0aff544f6.svg
Roei Herzig
user
roeiherz
658e4f96f5a209eeacb44051
Trevor Darrell
Unsupervised Universal Image Segmentation
Several unsupervised image segmentation approaches have been proposed which
eliminate the need for dense manually-annotated segmentation masks; current
models separately handle either semantic segmentation (e.g., STEGO) or
class-agnostic instance segmentation (e.g., CutLER), but not both (i.e.,
panoptic segmentation). We propose an Unsupervised Universal Segmentation model
(U2Seg) adept at performing various image segmentation tasks -- instance,
semantic and panoptic -- using a novel unified framework. U2Seg generates
pseudo semantic labels for these segmentation tasks via leveraging
self-supervised models followed by clustering; each cluster represents
different semantic and/or instance membership of pixels. We then self-train the
model on these pseudo semantic labels, yielding substantial performance gains
over specialized methods tailored to each task: a +2.6 AP^{box} boost
vs. CutLER in unsupervised instance segmentation on COCO and a +7.0 PixelAcc
increase (vs. STEGO) in unsupervised semantic segmentation on COCOStuff.
Moreover, our method sets up a new baseline for unsupervised panoptic
segmentation, which has not been previously explored. U2Seg is also a strong
pretrained model for few-shot segmentation, surpassing CutLER by +5.0
AP^{mask} when trained on a low-data regime, e.g., only 1% COCO
labels. We hope our simple yet effective method can inspire more research on
unsupervised universal image segmentation.
A novel unsupervised model, U2Seg, achieves high performance in instance, semantic, and panoptic segmentation through self-supervised learning and clustering.
unsupervised image segmentation STEGO CutLER panoptic segmentation Unsupervised Universal Segmentation U2Seg self-supervised models clustering AP PixelAcc COCO COCOStuff few-shot segmentation