Hi chaslie!



 chaslie:

If I understand the command BCEWithLogitsLoss, it uses a sigmoid function, so the $64k is why do i get values in the outut of BCE which are outside +/-1?


As Doosti explained, BCEWitLogitsLoss (in effect) passes your out
tensor through a sigmoid(), not its results, so it won’t be forcing its
results to be in the range [0.0, 1.0].
Note, however, that logically your data tensor – the labels you pass
to BCEWithLogitsLoss – should be in the range [0.0, 1.0].  These
labels are normally either the discrete values 0.0 or 1.0 (and thought
of as binary class labels) or they are continuous values in [0.0, 1.0]
(and thought of as the probability of your sample being in class-“1”).
If you respect this restriction, BCEWithLogitsLoss will not return
negative loss values – its loss values will range from 0.0 to inf.
(If you do not respect this restriction, you will be calculating a loss
that doesn’t make sense, your gradients won’t make sense, and
your model won’t train properly.)
The fact that the output you get from BCEWithLogitsLoss contains
negative values means that you are breaking this rule and passing
data values outside of [0.0, 1.0].  (But this is your fault, not the
fault of BCEWithLogitsLoss.)  On the other hand, the fact that the
output contains values greater than 1 is fine, and, in fact
BCEWithLogitsLoss diverges to inf for predictions that are
“completely wrong.”
Best.
K. Frank