Solved by ptrblck in post #2 
              
              
                You won’t be able to call loss.backward() inside a block, where gradient calculation is disabled via with torch.set_grad_enabled(False). 
You could of course enable it, but calculating gradients in the validation set (and updating the model) is a bad idea, as you are leaking the validation dataset i…