Solved by ptrblck in post #2 
              
              
                Yes, it’s possible. 
However, using FP16, you should take care of some possible pitfalls, e.g. imprecise weight updates or gradient underflow. NVIDIA recently published their mixed precision utilities for PyTorch named apex. 
On the one side you have the automatic mixed precision training using amp,…