albanD:

(ce_loss + additional_loss).backward()


In doing so, this 1 will happen as mentioned by @ptrblck. So, the layers before the custom layers will also have gradients of contrastive loss accumulated in addition to that of CE loss. But, I want to restrict this gradient accumulation (of contrastive + CE loss) within the custom layer.



 albanD:

Why can’t it be an additional term of your loss that you backprop as usual


Probably, this would justify why I am doing it this way.