Solved by ptrblck in post #2 
              
              
                The output of torch.argmax doesn’t have a backward function, so you are breaking the computation graph in: 
output = torch.argmax(output,dim=1)

Try to pass the model output directly to the criterion instead.