Say I have a training situation where I have two loss functions defined and want to update my gradients by backward pass over both. If I simply specify:
err1 = criterion1(...)
err2 = criterion2(...)
err = err1 + err2
err.backward()
optimizer.step()

Is it possible that one gradient update will overpower the other so that, in effect, only one loss really gets optimized since the other gradient update is so much smaller?
If so, can I normalize the gradients of the two before combining them and make sure they have similar impact on my update?