Deep Learning, Artificial Intelligence, Convolutional Network, Semi Supervised Learning, Self Supervised Learning.
cost and with the teacher output using consistency cost. After the weights of the student model have been updated with gradient descent, the teacher model weights are updated as an exponential moving average (EMA) of the student weights. Both model outputs can be used for prediction, but at the end