chirag1992m:

but, this error will propagate and explode along multiple layers which is happening in the trained model.


Do you have evidence for this? A small test script?
float precision guarantees only upto 1e-6 of accuracy practically.