Thank you Thomas for the explanation. The reason does indeed make sense.
I tried to verify your claim that the two methods (for fixed learning rate) are equivalent, but it seems like this can only be achieved by rescaling the velocity for the Torch scheme:
Let p_t be a current parameter. Let lr1, u1, and v1 be learning rate, momentum, and velocity for the original scheme, and lr2, u2, and v2 the learning rate, momentum, and velocity for the PyTorch version. Let G_{t} be the gradient at time t.
The original scheme goes: p_{t+1} = p_{t} - v1_{t+1} = p_{t} - u1 v1_{t} - lr1 G_{t+1}
The PyTorch scheme goes: p_{t+1} = p_{t} - lr2 v2_{t+1} = p_{t} - lr2 u2 v2_{t} - lr2 G_{t+1}
Equating the two expressions leads to
p_{t} - u1 v1_{t} - lr1 G_{t+1} = p_{t} - lr2 u2 v2_{t} - lr2 G_{t+1},
in other words
lr1 = lr2
and
u1 v1_{t} = lr2 u2 v2_{t}.
If the velocities in the two schemes were the same, i.e. v1 = v2, the last equation becomes u1 = lr2 u2 or u2 = u1/lr2. With a legit choice for learning rate and u1, this can easily lead to u2 > 1, which is forbidden. Then, the two iteration schemes cannot be equivalent.
So instead of having v1 = v2, one can take v1 = lr v2 and u1 = u2.
This means that the velocities in the two methods are scaled differently. Is this correct?