hm … I cannot reproduce this issue anymore. I think it was related to one or more bugs/misconceptions. E.g., I didn’t know that the ToTensor() transform in pytorchvision already scales pixels values from [0, 255] to [0, 1] so that I normalized input images twice, which had super small values after that (i.e., pixel/255/255), which probably caused the network not to learn with the suboptimal weight initialization and passing probabilities to the CrossEntropyLoss. Anyway, thanks for looking into that!