I have currently the following network with two losses:

I have a single optimizer and found that multiplying by a scalar one of the losses leads to better results such that:
L_total = lambda L1 + L2
I have the hypothesis that the lambda hyperparameter should only affect the shared representation. Therefore, I would like to “apply” it after both separated networks backpropagation. My questions are:

How to apply the lambda to the loss after each of the single networks’ backpropagation so that only the shared representation is affected by (lambda L1 + L2). Is multiplying the gradients that come from L1 by lambda enough?
Does that make any sense?

Thank you.