Solved by ptrblck in post #2 
              
              
                This wouldn’t be necessary, if you are not planning to finetune the pretrained models. 

We recommend to use the native amp implementation via torch.cuda.amp instead of apex/amp. Since there is no amp.initialize method, it should just work, but let us know if you encounter any issues. 

The amp sta…