Solved by mcarilli in post #3 
              
              
                tl;dr torch.cuda.amp is the way to go moving forward. 
We published Apex Amp last year as an experimental mixed precision resource because Pytorch didn’t yet support the extensibility points to move it upstream cleanly.  However, asking people to install something separate was a headache.  Extension…