Investigating the Semantic Robustness of CLIP-based Zero-Shot Anomaly Segmentation
<|reference_start|>Investigating the Semantic Robustness of CLIP-based Zero-Shot Anomaly Segmentation: Zero-shot anomaly segmentation using pre-trained foundation models is a promising approach that enables effective algorithms without expensive, domain-specific training or fine-tuning. Ensuring that these methods work across various environmental conditions and are robust to distribution shifts is an open problem. We investigate the performance of WinCLIP [14] zero-shot anomaly segmentation algorithm by perturbing test data using three semantic transformations: bounded angular rotations, bounded saturation shifts, and hue shifts. We empirically measure a lower performance bound by aggregating across per-sample worst-case perturbations and find that average performance drops by up to 20% in area under the ROC curve and 40% in area under the per-region overlap curve. We find that performance is consistently lowered on three CLIP backbones, regardless of model architecture or learning objective, demonstrating a need for careful performance evaluation.<|reference_end|>
@article{stangl2024investigating,
    title={Investigating the Semantic Robustness of CLIP-based Zero-Shot Anomaly
  Segmentation},
    author={Kevin Stangl, Marius Arvinte, Weilin Xu, Cory Cornelius},
    journal={arXiv preprint arXiv:2405.07969},
    year={2024},
    archivePrefix={arXiv},
    eprint={2405.07969},
    primaryClass={cs.CV cs.AI}
}