there is no option to disable such optimization related to singleton batch dimension, correct?

This is not an optimization, it is a limit of the floating point standard. Since you have limited precision, you have errors in your computations, and the more computations you perform, the bigger the error gets.