https://devblogs.nvidia.com/programming-tensor-cores-cuda-9/ 18 states that “Each Tensor Core performs 64 floating point FMA mixed-precision operations per clock (FP16 input multiply with full-precision product and FP32 accumulate, as Figure 2 shows)”.
Is it possible to multiply two fp16 tensors but get output in fp32? torch.mm with out argument of type fp32 produces an error: RuntimeError: Expected object of scalar type Half but got scalar type Float for argument #0 'result'
According to nvidia accumulation is done in fp32 so seems wasteful (in terms of performance) to return just fp16 as an output?
I’m aware of apex. Question here isn’t about end-to-end model training, but simply about matrix multiplication