Open-World Semantic Segmentation Including Class Similarity
<|reference_start|>Open-World Semantic Segmentation Including Class Similarity: Interpreting camera data is key for autonomously acting systems, such as autonomous vehicles. Vision systems that operate in real-world environments must be able to understand their surroundings and need the ability to deal with novel situations. This paper tackles open-world semantic segmentation, i.e., the variant of interpreting image data in which objects occur that have not been seen during training. We propose a novel approach that performs accurate closed-world semantic segmentation and, at the same time, can identify new categories without requiring any additional training data. Our approach additionally provides a similarity measure for every newly discovered class in an image to a known category, which can be useful information in downstream tasks such as planning or mapping. Through extensive experiments, we show that our model achieves state-of-the-art results on classes known from training data as well as for anomaly segmentation and can distinguish between different unknown classes.<|reference_end|>
@article{sodano2024open-world,
    title={Open-World Semantic Segmentation Including Class Similarity},
    author={Matteo Sodano, Federico Magistri, Lucas Nunes, Jens Behley, Cyrill
  Stachniss},
    journal={arXiv preprint arXiv:2403.07532},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.07532},
    primaryClass={cs.CV}
}