Since you don’t have to initialize the model for mixed-precision training in native AMP, you should wrap the model into DDP and apply the autocast and scaling as described in the examples.


I don’t know, if there is a clean way currently (and think we should add a utility function for it). At the moment you could check, if the new scale factor was updated after calling scaler.step(optimizer).


Yes, that should be possible, but please let us know, if you encounter any issues.