Hello Atharva!



 Atharva_Dubey:

Thanks a lot @KFrank.
I had read that BCEWithLogitsLoss() essentialy has a built sigmoid in it, so during the training it was alright, but when predicting for an individual input, I will have to use a sigmoid on the output.  Is my understanding correct?


It depends on what you mean by “predict.”
sigmoid() converts logits to probabilities.  sigmoid (0) = 1/2.
If you wish to predict the value of a binary variable, that is, say
whether your best guess for the value of the variable is 0 or 1,
then it is sensible to say that if the probability of the variable
being 1 is greater than 1/2, you will predict 1; otherwise you
will predict 0.
So, prediction = (probability >= 1/2).  But this is the
same as prediction = (logit >= 0).  So you don’t need
the sigmoid() or the probability to make this kind of prediction.
Simply take the output of your model (which is the output of a
Linear, and should be understood as a logit) and compare it to 0.
But if by “predict” you mean that you want to state the probability
of your variable being 1, then you do, of course, need to convert
the logit to a probability by passing it through a sigmoid().
When you compute the accuracy of your model, you usually
compare 0-or-1-valued predictions with 0-or-1-value targets,
so you don’t need sigmoid() – you just compare your logits
with 0 to get the 0-or-1 (False-or-True) prediction.
When you compute the loss for your model, you often use
cross-entropy, which works with probabilities, so the computation
needs to convert the logits to probabilities.  You can do this
yourself with an explicit sigmoid(), but for numerical reasons,
it is better to use BCEWithLogitsLoss which has (in effect) a
sigmoid() built into it.
Best.
K. Frank