I see. I considered just freezing part of the model and using one loss for that part, and a second loss for another part. But my concern is that after I backward the first loss, since the parameter has already been changed, will that affect the second backward call?