As far as I saw other implementations, I found that a seperate loss function is created as you mentioned. But those implementations have two seperate networks one for contrastive loss and then fine tuning with the other one for classification task by CE loss.
But in my case I need to proceed with only one network with contrastive loss included in backward of one layer so that the network learns the features of that particular layer and should update its weight accordingly. That’s why there are these terms in backward:
grad_weight += cont_loss_weight and grad_bias += cont_loss_bias
So, I have to stick to this approach  but I haven’t seen such implementation or thread regarding this.