googlebot:

Gradient norm is related to current goodness of fit, optimizer adjustments may not be needed, however parameter sharing during multi-task training may sometimes by problematic by itself.


Thanks for your reply! But I’m not sure about the meaning of “parameter sharing during multi-task training”. Here, I just add some new losses or increase the loss weight in a single-task model. The gradient norm increases. I’m not sure about the effect and the underlying meaning of gradient norm.