Hello Muster!



 wmpauli:

for some samples the target category was -1. Interestingly, when I try to replicate calling nn.CrossEntropyLoss directly on tensors on the CPU, I get an error message, if one of the targets is -1. When I run this on the GPU, I appear to get negative loss, rather than an error message.


I can reproduce this on pytorch 0.3.0 (except I don’t get a negative
loss).
If I have a target class label that is out of range, I get an error on the
cpu, but not on the gpu.
I’m a little surprised that the gpu lets this error pass, although perhaps
it forgoes checking for it in the interest of efficiency.
@ptrblck Do you think this is an issue or by design?
Here is a test script:
import torch
torch.__version__

predc = torch.autograd.Variable (torch.FloatTensor ([[0.5, -0.1, -0.2],[-0.1, -0.1, 0.4]]))
targc = torch.autograd.Variable (torch.LongTensor ([0, 2]))

predg = predc.cuda()
targg = targc.cuda()

torch.nn.functional.cross_entropy (predg, targg)
torch.nn.functional.cross_entropy (predc, targc)

targg[0] = -1
targc[0] = -1

torch.nn.functional.cross_entropy (predg, targg)
torch.nn.functional.cross_entropy (predc, targc)

Here is the output:
>>> import torch
>>> torch.__version__
'0.3.0b0+591e73e'
>>>
>>> predc = torch.autograd.Variable (torch.FloatTensor ([[0.5, -0.1, -0.2],[-0.1, -0.1, 0.4]]))
>>> targc = torch.autograd.Variable (torch.LongTensor ([0, 2]))
>>>
>>> predg = predc.cuda()
>>> targg = targc.cuda()
>>>
>>> torch.nn.functional.cross_entropy (predg, targg)
Variable containing:
 0.7550
[torch.cuda.FloatTensor of size 1 (GPU 0)]

>>> torch.nn.functional.cross_entropy (predc, targc)
Variable containing:
 0.7550
[torch.FloatTensor of size 1]

>>>
>>> targg[0] = -1
>>> targc[0] = -1
>>>
>>> torch.nn.functional.cross_entropy (predg, targg)
Variable containing:
 0.3972
[torch.cuda.FloatTensor of size 1 (GPU 0)]

>>> torch.nn.functional.cross_entropy (predc, targc)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\LisaBrown\Documents\admin\programs\Miniconda3\lib\site-packages\torch\nn\functional.py", line 1140, in cross_entropy
    return nll_loss(log_softmax(input, 1), target, weight, size_average, ignore_index, reduce)
  File "C:\Users\LisaBrown\Documents\admin\programs\Miniconda3\lib\site-packages\torch\nn\functional.py", line 1049, in nll_loss
    return torch._C._nn.nll_loss(input, target, weight, size_average, ignore_index, reduce)
RuntimeError: Assertion `cur_target >= 0 && cur_target < n_classes' failed.  at d:\pytorch\pytorch\torch\lib\thnn\generic/ClassNLLCriterion.c:87

Best.
K. Frank