Scenario:
The question comes from the fact that I am training a ResNet on CIFAR10 and the training loss reaches 0% at around epoch 100 and I still don’t match enough accuracy on the test set.
I don’t think I am overfitting since the validation loss keeps reducing as well. So I didn’t go for increasing the weight decay.
Question:
Any idea on how could I manage to kind of keep room for improvement and not reach the 100% accuracy while training so fast? @rasbt @ptrblck
I thought in increasing the shuffling but given your answer this is not an option any more 
Thank you!