OccFeat: Self-supervised Occupancy Feature Prediction for Pretraining BEV Segmentation Networks
<|reference_start|>OccFeat: Self-supervised Occupancy Feature Prediction for Pretraining BEV Segmentation Networks: We introduce a self-supervised pretraining method, called OccFeat, for camera-only Bird's-Eye-View (BEV) segmentation networks. With OccFeat, we pretrain a BEV network via occupancy prediction and feature distillation tasks. Occupancy prediction provides a 3D geometric understanding of the scene to the model. However, the geometry learned is class-agnostic. Hence, we add semantic information to the model in the 3D space through distillation from a self-supervised pretrained image foundation model. Models pretrained with our method exhibit improved BEV semantic segmentation performance, particularly in low-data scenarios. Moreover, empirical results affirm the efficacy of integrating feature distillation with 3D occupancy prediction in our pretraining approach. Repository: https://github.com/valeoai/Occfeat<|reference_end|>
@article{sirko-galouchenko2024occfeat:,
    title={OccFeat: Self-supervised Occupancy Feature Prediction for Pretraining
  BEV Segmentation Networks},
    author={Sophia Sirko-Galouchenko, Alexandre Boulch, Spyros Gidaris, Andrei
  Bursuc, Antonin Vobecky, Patrick P'erez, Renaud Marlet},
    journal={arXiv preprint arXiv:2404.14027},
    year={2024},
    archivePrefix={arXiv},
    eprint={2404.14027},
    primaryClass={cs.CV cs.LG}
}