The data type that is used is the one from the input Tensor. So that will depend on the type of your input.
I am still unsure how you identify that the FMA is the only cause of your issue.
The algorithms used on GPU (especially if you use cudnn) are wildly different from the CPU ones and the lack of associativity will break any hope of getting the same result no?