Hi @abhinav_hari
My guess is that they used a warming-up learning rate 0.01 only for the first 400 iterations, and then the training error was 80% as a result.
For warm-up strategy, torchvision’s example 1 code may help.
FYI, I reproduced the accuracy of ResNet-110 for CIFAR-10 even without the warm-up strategy, and the code + trained model weights are publicly available as part of torchdistill.
See this page for the reproduced result. Google Colab examples to reproduce the trends are also available here 1.