There is not dimension problem, its due to extreme values of inds_g variable that is causing out of  bound error in torch.gather(). I validated this by setting inds_g to zero, then code runs on gpu as well.
But I do not why on gpu values are computed to extreme.