Hi @ptrblck Recently I encountered an interesting thing. When I trained my model, I didn’t save states of GradScaler, but saved optimizer states. Later I found that if I resumed training by loading the checkpoint, the performance was higher than continuing the training. The only difference I spot is that the GradScaler is newly created in the resumed session. So is it possible that if we recreate/reset GradScaler at each epoch, the performance would be actually higher than using the old GradScaler? Thanks.