If so, why using  torch.gather  once can get determined results?

I don’t know for sure. But it might be that with your particular hardware, a single gather happen to most of the time return bit-wise exact results. While queuing more than one expose a different behavior because it puts the gpu in a different state than it was when you call only one?
Why do you need exact determinism? knowing that it won’t work as soon as you change machine/pytorch version/dependency version?