No, the GradScaler would scale the loss, thus also the gradients, but the gradients would be unscaled before the optimizer.step() operation is performed. The loss scaling factor also has a growth_factor and growth_interval which would increase it after growth_interval steps were performed without a decrease by growth_factor. In case the scaling factor is low right before saving the checkpoint it could potentially benefit from a large factor again.