From my understanding of how pytorch works, there are two main categories:

Functions that aggregate together more operations (and typically inherit from nn.Module and have learnable parameters)
Functions that define a new “black box” operation (and do not have learnable parameters)

For the first type of functions you only need to define the .forward() method, specifying which operations are being computed… The backward() method is computed implicitly by torch.autograd by “reversing” all the operations done on the Variable. Clearly, to be able to do this, all the transformations applied on your variable need to be a autograd.Function instance and implement the backward method. So you cannot call math.sqrt(input) and expect it to work, but have to use appropriate functions. Note that since torch overwrites operators like +, *, div etc. it will work in a lot of cases, but it’s good to keep in mind what is going on
The second type need to implement both forward and backward: autograd will treat it as a black box and to compute the gradient it will simply call .backward()
Now, back to your example: my understanding is that you are after the first “type” of function. The problem is that autograd cannot compute the gradient of your customLoss, because you have not implemented it. The binary_cross_entropy function does not define a gradient because it only performs operation on input that autograd knows and can differentiate. For example, if you do something like
def add1(input):
return input + 1
then you don’t need to define backward(), because autograd can compute the gradient of add1 automatically, because it understands what ‘+’ means. This understanding comes from having defined the add method in the Variable class so that it understands what operation is performing. But if in your custom function you perform some other operation that it does not know about, then it can’t compute the gradient and you need to provide to provide a “black box” function that inherits from Function and implements a .backward() method