Solved by ptrblck in post #2 
              
              
                You can find the gradient clipping example for torch.cuda.amp here. 
What is missing in your code is the gradient unscaling before the clipping is applied. 
Otherwise you would clip the scaled gradients, which could then potentially zero them out during the following unscaling.