Hi all. I am running an autoencoder type model with MSELoss at the end. The problem arose when I noticed that my training loss was in the order of 100k and my validation loss was around 0.8. I tried running validation code with the training set for a sanity check and was still seeing the dramatic difference. I also tried removing all my batchnorms for a sanity check, those didn’t do anything.
However when I removed the with torch.no_grad(): and ran the validation code, my errors started to make sense (for the validation data as well as training). This seems to be a pytorch problem however I’m not sure if I’m not using something correctly. I’ve used torch.no_grad() before and it worked fine, the only difference right now is that I usually use python3 whereas right now I’m using python2.7 with pytorch 0.4.1
For now I’ll remove the no_grad() scope however I would like to use it since it speeds up validation computation. Does anyone have any ideas?
Edit: Using miniconda2