You could try to use float64 for all calculations and check the stability again. This would make the execution of your model slower, but could be informative for debugging further.
How often do you need to accumulate the predictions and how many values do you have?
Could you post an example of such a prediction, which is changed due to the floating point errors?