Yes, its correct.
I don’t have any sample here, but I tested implementing a simple equation, then I passed multiple values to this equation, change the order with gather, applied another function and the backpropagated. Then I calculated the derivatives manually and compared with the pytorch output, so I could verify pytorch was considering the ordering while backpropagating.