Hi Maria!



 Maria_Fernanda_De_La:

Hi I’m sorry is this has an obvious answer, but I’m struggling a bit understanding the exponential loss function.
Why do you multiply the labels times 2.0 and subtract 1.0


In short,  2 * label - 1 maps a label equal to the values {0, 1}
to the values {-1, 1}, to better match your output (the “logit”)
running from -infinity to +infinity.
Some more details:
First, I don’t know if there is one single standard definition of
exponential loss function.  (There might be; I just don’t know.)
Anyway, I think of it as being:
   exp (mismatch between prediction and actual),

where exp (mismatch) is used to strongly amplify being wrong.
In the case of a binary classification problem, it is often convenient
to have (and you often do have) numerical class labels where
label = 0 indicates the “negative” class and label = 1
indicates the “positive” class.
To be concrete, let’s say that the input to your neural-network
model is 65,536 floating-point grayscale values that represent
the pixels in a 256x256 grayscale image.  Your model has a
single floating-point output.  Your known class labels are 0 and
1, where 1 indicates that the image is a picture of a bird, and
0 indicates that it is not a bird.  (“Not a bird” means that it might
be a picture of nothing,  or it might be a picture of something
other than a bird, but it’s not a bird.)
The number-one rule is that your output means whatever you
train it to mean, but we will understand your output as follows:
A large positive number means that your model really thinks
the image is a bird; a moderate positive number means probably
a bird; 0 means your model has no opinion – it might be a bird,
but, equally likely, might not; a negative number means it probably
isn’t a bird.  Algebraically larger (more positive or less negative)
means it’s more likely to be a bird, and algebraically smaller (more
negative or less positive) means less likely.
Now to your specific question:
Your label values are 0 and 1.  2 * label - 1 maps these two
values to -1 and 1.  When label = 0, exp (label * output)
would always be 1, regardless of the value of output.  So, for a
non-bird image (label = 0), your model would not be penalized
for returning a very large, positive output – “Yes, it’s really a bird” –
nor rewarded for returning a negative output – “No, it’s not a bird.”
Mapping {0, 1} to {-1, 1} fixes this.
If you have a sharp eye, you should now notice that I missed a
minus sign in the loss function I posted above.  It should actually
be:
def myExpLoss (logits, labels):
    return  (( -( (2.0 * labels.float() - 1.0) * logits ) ).exp()).mean()

With this minus sign, this loss penalizes (loss > 1) for being
wrong, and rewards (loss < 1) for being right.  (A bigger – more
positive – loss means your model is getting things wrong; a
smaller loss means your model is getting things right.)
Good luck.
K. Frank