I met the same problem on pytorch v1.0, I got auc of 0.73 on validation set but when I load the model, I only got auc of 0.51 on the same data. and I also called model.eval() before calculate the auc metric. I still don’t know why.