Thanks for the correction on this and sorry for the confusion!
I misunderstood the documentation saying

momentum – the value used for the running_mean and running_var computation.

Sounds like momentum is the value used to scale the so far accumulated running_* params.
With a default value of 0.1 my understanding doesn’t really make sense though.