torch.index_select does support backpropagation:
x = torch.randn(3, 3, requires_grad=True)
out = torch.index_select(x, 0, torch.tensor([0, 2]))
out.mean().backward()
print(x.grad)
> tensor([[0.1667, 0.1667, 0.1667],
          [0.0000, 0.0000, 0.0000],
          [0.1667, 0.1667, 0.1667]])