Solved by Eiphodos in post #4 
              
              
                Looking at the code you are trying to run, optimizer.zero_grad() is probably located in engine/train_one_epoch and you will also need to edit the forward function in the actual model. 
See Automatic Mixed Precision examples — PyTorch 1.7.0 documentation for use case examples. 
I want to point out th…