Solved by ptrblck in post #2 
              
              
                If you use reduction='sum' in your criterion, the gradients should be the same: 
model = nn.Linear(10, 2)
criterion = nn.NLLLoss(reduction='sum')

x = torch.randn(2, 10)
y = torch.empty(2, dtype=torch.long).random_(2)

# use whole batch
loss = criterion(model(x), y)
loss.backward()
model_grad_batch …