3.7.3 Going beyond classiﬁcation
While classiﬁcation is a commonly used performance metric for evaluating self-supervised
learning models, it is important to consider other types of vision tasks as well. Tasks such
as object detection and semantic segmentation have gained popularity as they require
models to learn more complex representations of visual information. Recent works Caron
et al. [2021], Zhou et al. [2022a], Bardes et al. [2022] have demonstrated the eﬀectiveness
of self-supervised learning for these tasks. However, a limitation is that there is currently
no standardized protocol for evaluating self-supervised models on these tasks. Various
evaluation methods exist, such as ﬁnetuning the encoder on a downstream task or using
the encoder as a feature extractor. Further research is needed to establish a standardized
evaluation protocol for these tasks in the context of self-supervised learning.
A Cookbook of Self-Supervised Learning