Continual Segmentation with Disentangled Objectness Learning and Class Recognition
<|reference_start|>Continual Segmentation with Disentangled Objectness Learning and Class Recognition: Most continual segmentation methods tackle the problem as a per-pixel classification task. However, such a paradigm is very challenging, and we find query-based segmenters with built-in objectness have inherent advantages compared with per-pixel ones, as objectness has strong transfer ability and forgetting resistance. Based on these findings, we propose CoMasTRe by disentangling continual segmentation into two stages: forgetting-resistant continual objectness learning and well-researched continual classification. CoMasTRe uses a two-stage segmenter learning class-agnostic mask proposals at the first stage and leaving recognition to the second stage. During continual learning, a simple but effective distillation is adopted to strengthen objectness. To further mitigate the forgetting of old classes, we design a multi-label class distillation strategy suited for segmentation. We assess the effectiveness of CoMasTRe on PASCAL VOC and ADE20K. Extensive experiments show that our method outperforms per-pixel and query-based methods on both datasets. Code will be available at https://github.com/jordangong/CoMasTRe.<|reference_end|>
@article{gong2024continual,
    title={Continual Segmentation with Disentangled Objectness Learning and Class
  Recognition},
    author={Yizheng Gong, Siyue Yu, Xiaoyang Wang, Jimin Xiao},
    journal={arXiv preprint arXiv:2403.03477},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.03477},
    primaryClass={cs.CV}
}