alex.veuthey:

One way to check if it “works” (the loss value decreases if it should decrease) is to plot it separately


The problem here, I could be wrong of course, is that the model might learn from the other losses, hence, the custom loss will still decrease/change. Thus, to overcome this, we might need to zerofy the other losses, prior to the backward pass, and only keep the custom loss and plot it. If the loss curve does not change, this means the custom loss is not used to update the model. This is because no gradients are generated to update the weights with backprop. Still as an alternative, and as noted by @LeviViana, we can zerofy the other losses and take a look at the gradients of the model.