Our examples page 638 demonstrates the use of FP16_Optimizer and Apex DistributedDataParallel.  Amp examples are coming soon, and Amp’s use is thoroughly discussed in its README 105.
Give Apex a try and let us know what you think!
sorry for double post, the forum page told me “new users may only post 2 links at a time” or something along those lines.