Yes indexing is fully differentiable in PyTorch!
You can see this if you print out a tensor that you just performed indexing (selecting) on: tensor(..., grad_fn=<SelectBackward>)
Obviously this only back-propagates into the indexed elements with the other elements receiving zero gradient. This is necessary with some techniques such as Policy Gradients however.
Also thanks again to @fmassa for the clean answer!