Hello Sina!



 sinaabdollahi:

The problem is a multiclass classification (just one class must be activated!):
…
and in the forward() method of the Model class, the return line is a 33-dimensional vector :
…
and with the following criterion:
loss = criterion(y_hat, labels.long())
where,
criterion = torch.nn.CrossEntropyLoss()
Then, it launches the following error:
**RuntimeError: multi-target not supported …
…
When I print the y_hat and the labels, I get the following result (batch_size=2):
tensor([[0.0126, 0.0251, 0.0233, 0.0266, 0.0150, 0.0274, 0.0565, 0.0332, 0.0490,
0.0282, 0.0231, 0.0313, 0.0362, 0.0102, 0.0111, 0.0923, 0.0161, 0.0842,
0.0192, 0.0235, 0.0163, 0.0274, 0.0200, 0.0190, 0.0223, 0.0361, 0.0098,
0.0532, 0.0126, 0.0279, 0.0771, 0.0254, 0.0085],
[0.0276, 0.0481, 0.0292, 0.0282, 0.0292, 0.0101, 0.0470, 0.0183, 0.0195,
0.0154, 0.0347, 0.0329, 0.0257, 0.0327, 0.0100, 0.0998, 0.0284, 0.0368,
0.0137, 0.0426, 0.0376, 0.0172, 0.0099, 0.0166, 0.0305, 0.0196, 0.0275,
0.0179, 0.0287, 0.0394, 0.0295, 0.0697, 0.0259]],
grad_fn=)
tensor([[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]])


The problem is that CrossEntropyLoss takes “labels” (often
called “targets”) that are integer class labels (in the range
[0, nClass), i.e., 0 <= label < nClass), rather than
one-hot encoded vectors.
Your example y_hat and labels correspond to a batch size
of two, and a number of classes nClass = 33.  Therefore a
batch of labels should be a tensor of shape (2, ), i.e., just
two class-label integers in the range [0, 33).  You are passing
CrossEntropyLoss a (one-hot encoded)labels vector of shape
(2, 33).  (Your y_hat has the correct shape, (2, 33), i.e.,
(batchSize, nClass), that is expected by CrossEntropyLoss.)
Edit:
Also, CrossEntropyLoss expects “raw scores” (logits) for its
predictions (your y_hat), not probabilities.  So you do not want
a softmax() for the output of your model.  Just use the output
of your last linear layer:
Z = self.linear_final(Z)
return Z

rather than:
Z = torch.nn.functional.softmax(self.linear_final(Z), dim=1)
return Z

Best.
K. Frank