Right… since it is doing copy twice. The cat version is only copying once.
If these indices are fixed, you may want to look into masked_select. http://pytorch.org/docs/master/torch.html#torch.masked_select 2.6k.