Hello Hwarang!



 Hwarang_Kim:

def myloss(data):
    if blah blah:
        loss = blah
    if blah blah:
        loss = blah
    return loss
loss = myloss(output)
loss.backward()

but how can my loss function be backpropagated?
Is my loss function differentiable?


In short, the conditional statement doesn’t break anything.
If loss inside of your myloss() function is calculated with pytorch
tensor operations (that have backward() implemented and are
differentiable), backpropagation through myloss() will work just fine.
So, to be concrete, let:
def myloss (data):
    if data[0][0] > 5.0:
        loss = 1.0 * (data**2).sum()
    else:
        loss = 2.0 * (data**3).sum()
    return loss

Mathematically speaking, myloss() will be differentiable everywhere
except at data[0][0] = 5.0, which is good enough.
In practice, if data[0][0] = 5.0, myloss() will take the second
branch and loss.backward() will calculate the gradient that
corresponds to loss = 2.0 * (data**3).sum().
Best.
K. Frank