Hello Georges!



 gquenot:

Hello,
I am trying to use BCEWithLogitsLoss() with additional dimensions but it crashes. The error messages are different in cases the additional axis has a size of 1 or 2:
…
RuntimeError: output with shape [8, 10, 1] doesn't match the broadcast shape [8, 10, 10]

…
RuntimeError: The size of tensor a (10) must match the size of tensor b (2) at non-singleton dimension 2

In both cases, I am unable to understand the error message. The x and y tensors have the same shape and the documentation says that there can be any number of additional dimensions after the batch one. Can anyone tell me what I got wrong?


Your use of BCEWithLogitsLoss looks correct to me.  I am unable
to reproduce the errors you are getting using pytorch version 0.3.0.
Note. 0.3.0 does not support the pos_weight argument to the
BCEWithLogitsLoss constructor, so maybe there’s a hint there.
Could you tell us what version of pytorch you are using?
Two things you might try:
First, try leaving out the pos_weight argument when you construct
criterion:
criterion = torch.nn.BCEWithLogitsLoss()

Second, does it work for you if you don’t use “additional dimensions?”
That is, try it without the groups dimension (no groups at all, not just
groups = 1).
Here is my version of your script, tweaked to run with pytorch 0.3.0,
hence no pos_weight (and no built-in one_hot()):
import torch
torch.__version__
torch.manual_seed (2020)

batch_size = 8
groups = 1
num_classes = 10

# pos_weight = (num_classes-1.0)*torch.ones([num_classes])
# criterion = nn.BCEWithLogitsLoss(pos_weight=pos_weight)
criterion = torch.nn.BCEWithLogitsLoss()

x = torch.rand (batch_size, num_classes, groups)
print (x.shape)

y = torch.zeros (batch_size).long().random_ (num_classes)
print (y.shape)
y = y.unsqueeze (1).expand (-1, groups)
print (y.shape)
# one-hot y
y = torch.zeros (batch_size, num_classes, groups).scatter_ (1, y.unsqueeze (1), 1.0)
print (y.shape)

print (x[0])
print (y[0])
criterion (x, y)

groups = 2

x = torch.rand (batch_size, num_classes, groups)
print (x.shape)

y = torch.zeros (batch_size).long().random_ (num_classes)
print (y.shape)
y = y.unsqueeze (1).expand (-1, groups)
print (y.shape)
# one-hot y
y = torch.zeros (batch_size, num_classes, groups).scatter_ (1, y.unsqueeze (1), 1.0)
print (y.shape)

print (x[0])
print (y[0])
criterion (x, y)

And here is the output:
>>> import torch
>>> torch.__version__
'0.3.0b0+591e73e'
>>> torch.manual_seed (2020)
<torch._C.Generator object at 0x000001A55AB46630>
>>>
>>> batch_size = 8
>>> groups = 1
>>> num_classes = 10
>>>
>>> # pos_weight = (num_classes-1.0)*torch.ones([num_classes])
... # criterion = nn.BCEWithLogitsLoss(pos_weight=pos_weight)
... criterion = torch.nn.BCEWithLogitsLoss()
>>>
>>> x = torch.rand (batch_size, num_classes, groups)
>>> print (x.shape)
torch.Size([8, 10, 1])
>>>
>>> y = torch.zeros (batch_size).long().random_ (num_classes)
>>> print (y.shape)
torch.Size([8])
>>> y = y.unsqueeze (1).expand (-1, groups)
>>> print (y.shape)
torch.Size([8, 1])
>>> # one-hot y
... y = torch.zeros (batch_size, num_classes, groups).scatter_ (1, y.unsqueeze (1), 1.0)
>>> print (y.shape)
torch.Size([8, 10, 1])
>>>
>>> print (x[0])

 0.4869
 0.1052
 0.5883
 0.1161
 0.4949
 0.2824
 0.5899
 0.8105
 0.2512
 0.6307
[torch.FloatTensor of size 10x1]

>>> print (y[0])

    1
    0
    0
    0
    0
    0
    0
    0
    0
    0
[torch.FloatTensor of size 10x1]

>>> criterion (x, y)
0.9552074257284403
>>>
>>> groups = 2
>>>
>>> x = torch.rand (batch_size, num_classes, groups)
>>> print (x.shape)
torch.Size([8, 10, 2])
>>>
>>> y = torch.zeros (batch_size).long().random_ (num_classes)
>>> print (y.shape)
torch.Size([8])
>>> y = y.unsqueeze (1).expand (-1, groups)
>>> print (y.shape)
torch.Size([8, 2])
>>> # one-hot y
... y = torch.zeros (batch_size, num_classes, groups).scatter_ (1, y.unsqueeze (1), 1.0)
>>> print (y.shape)
torch.Size([8, 10, 2])
>>>
>>> print (x[0])

 0.0668  0.2524
 0.9783  0.9895
 0.0449  0.4335
 0.2581  0.3636
 0.1338  0.0855
 0.9679  0.7406
 0.4356  0.2211
 0.9607  0.8944
 0.3491  0.3937
 0.3645  0.1764
[torch.FloatTensor of size 10x2]

>>> print (y[0])

    0     0
    0     0
    0     0
    0     0
    0     0
    0     0
    0     0
    1     1
    0     0
    0     0
[torch.FloatTensor of size 10x2]

>>> criterion (x, y)
0.9362296428531408

Maybe someone could try this with pos_weight on pytorch 1.x (or
whenever pos_weight was added).
Best.
K. Frank