Towards Modality-agnostic Label-efficient Segmentation with Entropy-Regularized Distribution Alignment
<|reference_start|>Towards Modality-agnostic Label-efficient Segmentation with Entropy-Regularized Distribution Alignment: Label-efficient segmentation aims to perform effective segmentation on input data using only sparse and limited ground-truth labels for training. This topic is widely studied in 3D point cloud segmentation due to the difficulty of annotating point clouds densely, while it is also essential for cost-effective segmentation on 2D images. Until recently, pseudo-labels have been widely employed to facilitate training with limited ground-truth labels, and promising progress has been witnessed in both the 2D and 3D segmentation. However, existing pseudo-labeling approaches could suffer heavily from the noises and variations in unlabelled data, which would result in significant discrepancies between generated pseudo-labels and current model predictions during training. We analyze that this can further confuse and affect the model learning process, which shows to be a shared problem in label-efficient learning across both 2D and 3D modalities. To address this issue, we propose a novel learning strategy to regularize the pseudo-labels generated for training, thus effectively narrowing the gaps between pseudo-labels and model predictions. More specifically, our method introduces an Entropy Regularization loss and a Distribution Alignment loss for label-efficient learning, resulting in an ERDA learning strategy. Interestingly, by using KL distance to formulate the distribution alignment loss, ERDA reduces to a deceptively simple cross-entropy-based loss which optimizes both the pseudo-label generation module and the segmentation model simultaneously. In addition, we innovate in the pseudo-label generation to make our ERDA consistently effective across both 2D and 3D data modalities for segmentation. Enjoying simplicity and more modality-agnostic pseudo-label generation, our method has shown outstanding performance in fully utilizing all unlabeled data points for training across ...<|reference_end|>
@article{tang2024towards,
    title={Towards Modality-agnostic Label-efficient Segmentation with
  Entropy-Regularized Distribution Alignment},
    author={Liyao Tang, Zhe Chen, Shanshan Zhao, Chaoyue Wang, Dacheng Tao},
    journal={arXiv preprint arXiv:2408.16520},
    year={2024},
    archivePrefix={arXiv},
    eprint={2408.16520},
    primaryClass={cs.CV}
}