Deep Learning, Artificial Intelligence, Convolutional Network, Semi Supervised Learning, Self Supervised Learning.
first 100000 training steps As expected, the EMA-weighted models (blue and dark gray curves in the bottom row) give more accurate predictions than the bare student models (orange and light gray) after an initial period. Using the EMA-weighted model as the teacher improves results in the