Discriminative self-supervised learning. The second line of work, closer to ours, is using discriminative
signals between images or groups of images to learn features. This family of methods has roots in early
deep learning work (Hadsell et al., 2006) but became popular with the emergence of instance classiﬁcation
methods (Dosovitskiy et al., 2014; Bojanowski & Joulin, 2017; Wu et al., 2018). Several improvements
were made based either on instance-level objectives (Hénaﬀ et al., 2019; He et al., 2020; Chen & He, 2020;
Chen et al., 2020; Grill et al., 2020; Caron et al., 2021) or clustering (Caron et al., 2018; Asano et al.,
2020; Caron et al., 2020). These methods provide performant frozen features on standard benchmarks like
ImageNet (Russakovsky et al., 2015), but they are hard to scale to larger model sizes (Chen et al., 2021). In

3
DINOv2- Learning Robust Visual Features without Supervision