typically lead to a signiﬁcant drop in the quality of the features. This is explained by the lack of control
over the data quality and diversity, which are essential to produce good features.
In this work, we explore if self-supervised learning has the potential to learn all-purposed visual features if
pretrained on a large quantity of curated data. We revisit existing discriminative self-supervised approaches
that learn features at both the image and patch level, such as iBOT (Zhou et al., 2021), and we reconsider
some of their design choices under the lens of a larger dataset. Most of our technical contributions are tailored
toward stabilizing and accelerating discriminative self-supervised learning when scaling in model and data
sizes. These improvements make our approach around 2× faster and require 3× less memory than similar
discriminative self-supervised methods, allowing us to leverage longer training with larger batch sizes.
DINOv2- Learning Robust Visual Features without Supervision