overall feature representation [23], or that (under strong assumptions) it can identify
the latent structure in a hypothesized data-generating process, akin to nonlinear ICA
[Khemakhem et al., 2020]. In Wang and Isola [2020], Theorem 1 shows that contrastive
learning with an RBF kernel (an expressive map of features into a higher dimensional
space) converges to a uniform distribution on the sphere with matched pairs.
Uniﬁed contrastive losses. Tian [2022] uniﬁed contrastive losses as minimizing a
general family of loss functions Lφ,ψ, where φ and ψ are monotonously increasing and
diﬀerentiable scalar functions
A Cookbook of Self-Supervised Learning