regardless of the method, you cannot have the resulting Tensor share the same memory as the source Tensor.
The reason is because you aren’t asking for a slice of the original tensor, but you’re asking for particular indicees of the original Tensor. Only slices can be viewed on the memory of the original Tensor.
Your first way is the efficient version of doing things.
Alternatively, do see if this is faster (probably not, but worth a try):
W[ index.view(-1), :].view(index.size(0), index.size(1), 1024)