Open-Vocabulary Object Detectors: Robustness Challenges under Distribution Shifts
<|reference_start|>Open-Vocabulary Object Detectors: Robustness Challenges under Distribution Shifts: The challenge of Out-Of-Distribution (OOD) robustness remains a critical hurdle towards deploying deep vision models. Vision-Language Models (VLMs) have recently achieved groundbreaking results. VLM-based open-vocabulary object detection extends the capabilities of traditional object detection frameworks, enabling the recognition and classification of objects beyond predefined categories. Investigating OOD robustness in recent open-vocabulary object detection is essential to increase the trustworthiness of these models. This study presents a comprehensive robustness evaluation of the zero-shot capabilities of three recent open-vocabulary (OV) foundation object detection models: OWL-ViT, YOLO World, and Grounding DINO. Experiments carried out on the robustness benchmarks COCO-O, COCO-DC, and COCO-C encompassing distribution shifts due to information loss, corruption, adversarial attacks, and geometrical deformation, highlighting the challenges of the model's robustness to foster the research for achieving robustness. Project page: https://prakashchhipa.github.io/projects/ovod_robustness<|reference_end|>
@article{chhipa2024open-vocabulary,
    title={Open-Vocabulary Object Detectors: Robustness Challenges under
  Distribution Shifts},
    author={Prakash Chandra Chhipa, Kanjar De, Meenakshi Subhash Chippa, Rajkumar
  Saini, Marcus Liwicki},
    journal={arXiv preprint arXiv:2405.14874},
    year={2024},
    archivePrefix={arXiv},
    eprint={2405.14874},
    primaryClass={cs.CV}
}