Solved by ptrblck in post #2 
              
              
                If the first iteration creates NaN gradients (e.g. due to a high scaling factor and thus gradient overflow), the optimizer.step() will be skipped and you might get this warning. 
You could check the scaling factor via scaler.get_scale() and skip the learning rate scheduler, if it was decreased. I th…