Thanks for sharing the links. Adding some of my own favorites and others:

NeurIPS tutorial on self-supervision by Lilian Weng and Jong Wook Kim: Self-Supervised Learning: Self-Prediction and Contrastive Learning (slides 1)
A better minimal implementation of SimCLR with lots of cool stuff: Semi-supervised image classification using contrastive pretraining with SimCLR 1

SimSiam blog: Self-supervised contrastive learning with SimSiam (originally from FAIR)
Masked Image Modeling with Autoencoders (arguably the simplest one): Masked image modeling with Autoencoders (with @ariG23498, originally from FAIR)
Interview with Ishan Misra: #55 Self-Supervised Vision Models (Dr. Ishan Misra - FAIR). - YouTube.

Using self-supervision in a supervised setting:

https://arxiv.org/abs/2006.10803
Efficient Training of Visual Transformers with Small Datasets | OpenReview