• Image-level objective (Caron et al., 2021). We consider the cross-entropy loss between the
features extracted from a student and a teacher network. Both features are coming from the class
token of a ViT, obtained from diﬀerent crops of the same image. We learn the parameters of the
student and build the teacher with an exponential moving average of past iterates (He et al., 2020).

• Patch-level objective (Zhou et al., 2021). We randomly mask some of the input patches given
to the student, but not to the teacher. We then add a cross-entropy loss between the patch features
of both networks on each masked patch. This loss is combined with the image-level loss.

• Untying head weights between both objectives. We observe that tying the weights associated
with both objectives makes the model underﬁt at the patch-level while overﬁtting at the image-level.
Untying these weights resolves this issue and improve the performances at both scales.
DINOv2- Learning Robust Visual Features without Supervision