ngimel:

Please see here for tips on training with mixed precision http://docs.nvidia.com/deeplearning/sdk/mixed-precision-training/index.html


Those tips are very interesting. Has anyone got examples of implementing that in pytorch? Are there any examples of successfully training in half precision with pytorch - especially of standard architectures like resnet?