Implementation Details. The deduplication and retrieval stages of our pipeline rely on the Faiss li-
brary (Johnson et al., 2019) to eﬃciently index and compute batch searches of nearest embeddings.
In
particular, we heavily leverage its support for GPU-accelerated indices, using inverted ﬁle indices with prod-
uct quantization codes (Jegou et al., 2010). The whole processing is distributed on a compute cluster of 20
nodes equipped with 8 V100-32GB GPUs and takes less than two days to produce the LVD-142M dataset.

4 Discriminative Self-supervised Pre-training

We learn our features with a discriminative self-supervised method that can be seen as a combination of
DINO and iBOT losses with the centering of SwAV (Caron et al., 2020). We also add a regularizer to spread
features and a short high-resolution training phase. We rapidly introduce each of these approaches, but more
details can be found in the related papers, or in our open-sourced code.
DINOv2- Learning Robust Visual Features without Supervision