Looking at the indexing code 8, it moves the index to the same device as the indexed tensor lives in. As such it would appear be more efficient to do it in advance.
You could always time it (but don’t forget to call torch.cuda.synchronize() before you take the start/finish time), too. Indexing should be a bit faster if you have the index tensor on the GPU.
Best regards
Thomas