We recommend to use the native mixed-precision utility via torch.cuda.amp 13 as described here 28. New features, such as the compatibility with 3rd party repositories (transformers in this case), won’t land in apex/amp, but in native amp instead.