Hello Maria!



 Maria_Fernanda_De_La:

I was looking through the documentation and I was not able to find the standard exponential loss function. Is there a simple way to implement my own exponential function.


Sure.  And if you use normal tensor operations, autograd
will work for you.
Let’s say you have a binary classification problem and
logits are your predictions for a batch (running from
-infinity to +infinity) and labels are your known class
labels for that batch (equal to 0 or 1).
Then:
import torch

def myExpLoss (logits, labels):
    return  (((2.0 * labels.float() - 1.0) * logits).exp()).mean()

logits = torch.tensor ([-0.2, 1.3, .3], requires_grad = True)
labels = torch.tensor ([0, 0, 1])

myExpLoss (logits, labels)

[Edit:  The  above expression for myExpLoss is missing a minus
sign, and this causes the loss to incorrectly increase for more
accurate predictions.  I haven’t edited in the correction above,
but the correct expression appears in my next post, below.]
This yields:
>>> myExpLoss (logits, labels)
tensor(0.9479, grad_fn=<MeanBackward1>)

so you can see that autograd is working.
[Edit:  This loss value, 0.9479, is less than 1, incorrectly indicating
the the predictions are more right than wrong.  Although the first
and last predictions are right, they are only mildly right, while
the second prediction is strongly wrong, making the predictions
overall more wrong than right.  The correct value of the loss (given
by the corrected expression in my next post) is 1.7429.]
This version returns the per-sample exponential loss averaged
over the batch.
As an aside, you might not want to use an exponential loss.
Depending on your problem, some other loss function might
work better.
Good luck.
K. Frank