idea of exponential moving averages provide training stability that can even be used in
non student-teacher frameworks such as SimCLR. Speciﬁcally, they show applying EMA
updates to the projector of SimCLR can boost performance.
A Cookbook of Self-Supervised Learning