The GradScaler shouldn’t add this massive overhead, as it’ll check for invalid gradients and skip the optimiyer.step() if necessary. Could you post a minimal code snippet to reproduce this issue, so that we could debug it, please?