2.4 The Canonical Correlation Analysis Family:

VICReg/BarlowTwins/SWAV/W-MSE

The SSL canonical correlation analysis family originates with the Canonical Correlation
Framework (CCA) [Hotelling, 1992]. The high-level goal of CCA is to infer the relationship
between two variables by analyzing their cross-covariance matrices. Speciﬁcally, let
X ∈ RD and Y ∈ RD. The CCA framework seeks two transformations U = fx(X) and

13

V = fy(Y ) such that

N(cid:88)

L = − N(cid:88)
N(cid:88)
(cid:123)(cid:122)

1
N

n=1

n=1

(cid:104)Un, Vn(cid:105),

1
N

(cid:124)

(cid:125)

such that 1
N

(cid:124)

Un =

,
Vn = 0

U T U =

V T V = I

,

n=1
zero-mean representations

identity covariance representations

1
N

(cid:123)(cid:122)

(cid:125)

(13)

2

2

− 1
y
A Cookbook of Self-Supervised Learning