Solved by ptrblck in post #2 
              
              
                It seems to work: 
criterion = nn.CrossEntropyLoss(ignore_index=-1)

output = torch.randn(2, 10, requires_grad=True)
target = torch.tensor([-1, 1])

loss = criterion(output, target)
loss.backward()
print(loss, output.grad)
> tensor(1.4096, grad_fn=<NllLossBackward>) 
tensor([[ 0.0000,  0.0000,  0.00…