I think the conversion to float64 is necessary for the transformation.
However, afterwards I would transform it to float32, as you have already done.
I think you could start experimenting with the model now.