custom CUDA extensions = ?
Googled for any existing solutions for PyTorch, could only find some mentions of FP16 for inference in TensorFlow. Here https://blog.tensorflow.org/2019/06/high-performance-inference-with-TensorRT.html 7 and https://www.slideshare.net/VadimSolovey/gan-training-with-tensorflow-and-tensor-cores 4 for example.