Yuri, you are correct. Cublas allows returning result of the multiplication of 2 fp16 matrices in fp32, however, pytorch does not support this option. Pytorch could allow this if there is a compelling use case.