Hello Mona!
You are probably passing values for target (your labels) that are
either negative or larger than one to MultiLabelSoftMarginLoss,
causing it to return invalid negative values.



 Mona_Jalal:

criterion = nn.MultiLabelSoftMarginLoss()
...
        loss = criterion(outputs, labels.float())

The losses are:
Epoch [1/15], Step [5/17], Loss: -19443.7852
...
Epoch [15/15], Step [15/17], Loss: -4690499.0000



Conceptually, MultiLabelSoftMarginLoss should never return a
negative loss value.  Its input (the outputs of your model) should
be logits that run from -inf to inf that are, in effect, converted to
probabilities inside of MultiLabelSoftMarginLoss.  (This part is
correct in your code as your model returns the result of its last
Linear layer for your outputs.)
Its target (your labels), however, should be probabilities that run
from 0.0 to 1.0*.  If the target values are outside of this range, you
violate the requirements of MultiLabelSoftMarginLoss, which can
then return (invalid) negative values, and your training will likely diverge
(which it appears to be doing).
Check that your labels lie within [0.0, 1.0], and if not, ask what they
mean conceptually, and how (and whether) they can be converted to
probability-like values.
*)  The documentation for MultiLabelSoftMarginLoss 3 states that
 y[i]∈{0,  1},

that is, that y (the target) is equal to either 0 or 1.  But I believe
that MultiLabelSoftMarginLoss is meaningful for values of y that
range from 0.0 to 1.0, and the code accepts such values.
Best.
K. Frank