I don’t think you can force the order of operation to be constant, at least I’m not aware of any way.
How large are your activation values, when you have the large error?
I assume you would like to get rid of the 0.5 in absolute difference.