I found a case In https://pytorch.org/docs/master/notes/amp_examples.html#typical-mixed-precision-training 4.

If your network has multiple losses, you must call [ scaler.scale ] on each of them individually.
…
scaler.scale(loss0).backward(retain_graph=True)
scaler.scale(loss1).backward()

It seems that the gradients back propogated from the two losses that scaled by different scaling parameters are just simply accumulated. Can the scaler unscales the two set of gradients with their corresponding scaling parameters when it comes to parameter updating?