Solved by ptrblck in post #2 
              
              
                You are creating a new tensor as your loss, thus detaching the original output of your criterion from the computation graph: 
loss = torch.tensor(criterion(output_var, y_batch), requires_grad=True)

Instead just call: 
loss = criterion(output, y_batch)
loss.backward()

Note, that nn.CrossEntropyLoss …