In that case, you can use the nightly version of pytorch an use the new inputs argument to the .backward() function:
net.zero_grad()
ce_loss.backward()
additional_loss.backward(inputs=net.your_contrastive_layer.parameters())
opt.step()