Hi Ninoslav!



 Legolas:

I am trying to treat it as a multi-class problem (for the initial set of four classes, ‘AC’, ‘AD’, ‘BC’, ‘BD’) but would like to include ‘impact’ of class components.


I have a few comments:
First, to reiterate, since you want to “look inside” you classes, drilling
down into what you call the “class components,” it seems unnatural to
me to treat this as a single-label, four-class problem rather than a
multi-label, two-class problem.  By using your four classes, you’ve
“hidden” their components that you want access to, just making things
harder.
Now some thoughts about your approach:
The most common approach for a (single-label) four-class problem
would be to have the final Linear layer of your model output four
raw-score logits that you then pass into CrossEntropyLoss.  (The
ground-truth target passed into CrossEntropyLoss will be a single
integer class label (for each sample in the batch) that takes on the
values {0, 1, 2, 3}.)
You can convert the logits into probabilities for your four classes by
passing them through softmax().  (CrossEntropyLoss has, in effect,
softmax() built in.)
Note, the predictions coming out of your model are logits (that can be
converted to probabilities) specifically because your trained them to be
by using CrossEntropyLoss as the loss criterion you backpropagate.
Were you to use some other loss criterion for training the meaning of
your model outputs would likely change.
Nonetheless, let’s work under the assumption that your model outputs
remain logits even as we modify the loss criterion.
So:  Convert your model outputs to probabilities, and compute the
probabilities for your class “components”:
P(A) = P(AC) + P(AD)
P(B) = P(BC) + P(BD)
P(C) = P(AC) + P(BC)
P(D) = P(AD) + P(BD)

Note that P(A) + P(B) = 1 (which is consistent because the
first component is either in state A or state B), and, similarly,
P(C) + P(D) = 1.
Now consider your ground-truth target:  target = 1, for example,
means class AD, so 100% probability of the first component being
in state A (so 0% probability in state B) and 0% probability of the
second component being in state C (so 100% probability in state D).
You now have the both predicted and ground-truth probabilities of your
first component being in state A so you can use the mismatch between
these two probabilities (for example, the cross entropy) as your loss
criterion for the first component of your class, and weight it with alpha.
Similarly, you could use the mismatch between the predicted and
ground-truth probabilities for your second component being in state
C as your second-component loss criterion, and weight it with beta.
You can now train your model with this combined alpha and beta
loss criterion.  Problem solved!
However …
What you’ve actually done here is built a (weighted) multi-label
binary-cross-entropy loss criterion for the case where your multi-label
predictions are encoded in an unusual way.
How can the four outputs of the final Linear layer of your model turn
into multi-label predictions (with an unusual encoding) instead of being
the four conventional single-label predictions for your four classes?
Because that’s what you’ve trained them to be.  (The first rule of neural
networks is that the output of your model means what you train it to
mean.)
There’s nothing wrong nor inconsistent about doing things this way
(although it does seem like you would be standing on your head), but
the meaning of your model’s predictions is no longer the same as
if you had trained it as a conventional single-label, four-class model.
Instead, your model is making multi-label, two-class predictions, albeit
encoded in an unusual way.
Best.
K. Frank