To use TensorCores, you would have to apply some operations in FP16. You could transform (some) operations manually via .half() or I would recommend to use our mixed-precision approach in apex 4.
We are currently working on a direct PyTorch integration.