Active Label Correction for Semantic Segmentation with Foundation Models
<|reference_start|>Active Label Correction for Semantic Segmentation with Foundation Models: Training and validating models for semantic segmentation require datasets with pixel-wise annotations, which are notoriously labor-intensive. Although useful priors such as foundation models or crowdsourced datasets are available, they are error-prone. We hence propose an effective framework of active label correction (ALC) based on a design of correction query to rectify pseudo labels of pixels, which in turn is more annotator-friendly than the standard one inquiring to classify a pixel directly according to our theoretical analysis and user study. Specifically, leveraging foundation models providing useful zero-shot predictions on pseudo labels and superpixels, our method comprises two key techniques: (i) an annotator-friendly design of correction query with the pseudo labels, and (ii) an acquisition function looking ahead label expansions based on the superpixels. Experimental results on PASCAL, Cityscapes, and Kvasir-SEG datasets demonstrate the effectiveness of our ALC framework, outperforming prior methods for active semantic segmentation and label correction. Notably, utilizing our method, we obtained a revised dataset of PASCAL by rectifying errors in 2.6 million pixels in PASCAL dataset.<|reference_end|>
@article{kim2024active,
    title={Active Label Correction for Semantic Segmentation with Foundation Models},
    author={Hoyoung Kim, Sehyun Hwang, Suha Kwak, Jungseul Ok},
    journal={arXiv preprint arXiv:2403.10820},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.10820},
    primaryClass={cs.CV}
}