Hi Ninoslav!



 Legolas:

The first part of the label is always ‘A’ or ‘B’ and the second is always ‘C’ or ‘D’. So, there are only four possible classes: ‘AC’, ‘AD’, ‘BC’, BD’.


You should understand your problem to be a multi-label, multi-class
problem in the following sense:
It is multi-class because you have have multiple classes – in your case,
two classes, “emotion” and “positivity.”  And is is multi-label because
each sample is given not just one class label (i.e., “emotion” or
“positivity,” but not both) but rather multiple labels, potentially one for
each of your two classes.
That is, each of your samples can be labelled with neither, one, or
both of your two classes.  Said in another (but equivalent) way, each
sample carries two binary labels:  first binary label – “no-emotion”
vs. “yes-emotion”; and second binary label – “no-positivity” vs.
“yes-positivity.”
(It is true that you can structure this as a single-label, four-class problem
with the four labels being your ‘AC’, ‘AD’, ‘BC’, ‘BD’, but it is generally
better to structure this kind of problem as a multi-label problem.)
The preferred loss function will be BCEWithLogitsLoss 1.  Note that this
loss function explicitly supports “the case of multi-label classification.”


I need multiclass classification to four classes, ‘AC’, ‘AD’, ‘BC’, BD’, but I need a loss function that will include and learn/implement the relation between components A-B/C-D.


As noted above, you’ll be better of with two classes, but use the
multi-label structure.  Just to be clear, your “A-B” is the first of two
binary labels and “C-D” is the second.


the penalization factor for the wrong prediction of the first label should be alpha and the second beta.


Here is one benefit of using BCEWithLogitsLoss with the multi-label
approach.  Use BCEWithLogitsLoss's weight constructor-argument
with a tensor, [[alpha, beta]] (of shape [1, 2]).  (The leading
singleton dimension of weight will be broadcast along the batch
dimension, so that you’ll be using the same class weights for each
sample in batch.)  This will weight the (binary-cross-entropy) contribution
of your “emotion” class by alpha in your loss function and weight
“positivity” with beta.


It would be great to learn these parameters alpha and beta.


I don’t think learning alpha and beta really makes sense.
Maybe you care more about getting “emotion” right than “positivity.”
Or maybe the other way around.  I don’t see how your network can
know this without you telling it.  You tell the network how much you
care about “emotion” vs. “positivity” with your alpha and beta
parameters, so the network doesn’t have any way to “learn” this
preference of yours.


or it’s possible to implement just through the loss function (preferable)?


Yes, as outlined above, using just a loss function – specifically the
multi-label case of BCEWithLogitsLoss – it is possible – and likely
the best way – to implement your classifier.
(Just to be sure, I used your two-class example – “emotion” and
“positivity” – for simplicity and to follow along with your post.  But
there is nothing about this scheme that is restricted to two classes.
You could, for example, label your natural-language phrases with five
features, say “emotion,” “positivity,” “certainty,” “conciseness,” and
“understandability,” and use BCEWithLogitsLoss to build a multi-label,
five-class classifier.)
Good luck.
K. Frank