Hi Log!



 logarith:

I also have output tensors with the size of 60 classes, which are not mutual exclusive. There are always 10 classes one and the others zero.
network = torch.nn.Sequential(
            ...
            torch.nn.Linear(150, 60),            
        )

       loss_function = torch.nn.MultiLabelSoftMarginLoss()

It seems to me that it predicts a lot of negative values, what I dont understand.


Because the output of your model is the output of your last Linear
layer, you are predicting raw-score logits.  A logit value that is less
than zero corresponds to a predicted probability less than one half.
Typically a probability of less than one half for the “1” state when
interpreted as a hard “yes-no” prediction would be taken to be a
“0”-state prediction (and greater than one half would be the “1” state).
Many more of your “output-tensor” target values are 0's than are
1's, so if you weight each individual target value equally in loss
function, your model can train to do a good job on the loss function
by preferentially predicting 1's (that is, predicting negative logits),
regardless of the input data.
The common approach to addressing this is to weight your
less-frequent 1 target values more heavily in your loss function.
Note that BCEWithLogitsLoss 1 is essentially the same as
MultiLabelSoftMarginLoss but has a pos_weight argument
that you can pass to its constructor.
You say that you have 60 classes, and that any given sample target
has 10 classes in the 1 state and 50 in the 0 state.  If all of your
classes are about equally likely to be in the 1 state, you could use
the same pos_weight for all of them.  A reasonable value would be
pos_weight = n_negative / n_positive.  So:
loss_function = torch.nn.BCEWithLogitsLoss (pos_weight = torch.tensor ([5.0]))

If the likelihood of your different classes having target value 1 are
not all broadly similar, then you would pass in a tensor of length 60
for your pos_weight, that is, a different pos_weight value for each
class.
Best.
K. Frank