(cid:112)(effective_batch_size) / 256.

3.8.2 Even Faster Training with FFCV and Other Speedups
Since most join-embedding SSL methods requires diﬀerent set of handcrafted data aug-
mentation, data processing can become a real bottleneck when training SSL models. Some
approaches4 have used DALI as an alternative data loader to pytorch vision while some
other have relied on FFCV-SSL5 which is based on the FFCV library [Leclerc et al., 2022].
FFCV-SSL [Bordes et al., 2023a] shows that one can train SimCLR on ImageNet in less
than 2 days on a single GPU or in just a few hours using 8 GPUs (Figure 15).
A Cookbook of Self-Supervised Learning