Thank you for your help! Indeed I gained good speedup.
What if we level up the problem to include multiple batches (say 8) instead of just one. Noting that each batch can have different number of effective indicies. the ineffictive indicies can be filled with dummy data like (-1). Example

indices_0 = torch.tensor(([[1,2],[0,0],[1,2],[1,1],[1,0],[0,0],[1,2],[2,2],[2,2],[0,0]]))
indices_1 = torch.tensor(([[1,0],[0,0],[-1,-1],[-1,-1],[1,1],[0,0],[1,0],[0,0],[0,1],[0,0]]))
indices_2 = torch.tensor(([[-1,-1],[0,0],[3,1],[0,0],[0,0],[1,1],[1,1],[1,1],[1,2],[2,0]]))
...
indices_7 = torch.tensor(([[2,2],[0,1],[2,2],[1,1],[1,2],[0,0],[1,0],[1,2],[2,1],[0,1]]))

Here [-1,-1] is to be ignored. I indicated the ineffective indices with [-1,-1] just to spot them out. I can eliminate them from the tensor, but then I wont be able to put them in one tensor of shape (8,10,2) as they are of different lengths, unless there is a way around this too.
Those are 8 indices each representing a batch. The shape of all tensors will be as follows:
Whenever there is an ineffective (not to be counted) index then its corresponding data value will be [-1,-1,…,-1]. Again the -1’s are there to make all tensors of equal lengths.
result = torch.full((8, 32, 256, 256), -1).to(device)
data = torch.rand((8,32,40000))
indices shape is: (8, 10, 2)