  Self-supervised learning (SSL) is a growing torrent that has recently
transformed machine learning and its many real world applications, by learning
on massive amounts of unlabeled data via self-generated supervisory signals.
Unsupervised anomaly detection (AD) has also capitalized on SSL, by
self-generating pseudo-anomalies through various data augmentation functions or
external data exposure. In this vision paper, we first underline the importance
of the choice of SSL strategies on AD performance, by presenting evidences and
studies from the AD literature. Equipped with the understanding that SSL incurs
various hyperparameters (HPs) to carefully tune, we present recent developments
on unsupervised model selection and augmentation tuning for SSL-based AD. We
then highlight emerging challenges and future opportunities; on designing new
pretext tasks and augmentation functions for different data modalities,
creating novel model selection solutions for systematically tuning the SSL HPs,
as well as on capitalizing on the potential of pretrained foundation models on
AD through effective density estimation.

Machine Learning