antspy,
Thanks again for your extremely helpful insight!
From what you’ve written it sounds like if my custom loss function is built on simple operations then I will not need to define a custom backward() operation.
To make things a little less abstract let’s assume that I have two tensors A, B where A and B are both nxm.  Suppose my custom loss function is something like sum(A - B).  Could I simply implement this as a standard python function, or would I need to do something more complicated?
For example, could i just do something like this?
def custom_Loss(A,B):
return sum(A - B)
loss = custom_Loss(a,b)
loss.backward()