The shift from DML to what is now referred to as SSL might have occurred when Sohn
[2016] introduced the (N+1)-tuple loss, a loss similar to the contrastive predictive coding
(CPC) loss from [Oord et al., 2018]. The use of other sample positive views as the negative
view of other pairs is introduce as an eﬃcient strategy coined N-pair-mc loss. Ni et al.
[2021b] shows that contrastive learning is a special case of meta-learning, and existing
meta-learners can be directly applied to SSL with competitive performance. CPC was
extended to images in [Henaﬀ, 2020]. A key ingredient in CPC was the introduction of the
InfoNCE loss described in 3 Oh Song et al. [2016], which became central in SSL.
A Cookbook of Self-Supervised Learning