Solved by ptrblck in post #2 
              
              
                Indexing the tensor might work. 
I assume the initialization of k_block should be outside the first loop? 
Otherwise, you would throw away the first results. 
Based on this assumption, this code should work: 
# Setup
B, H, K = 2, 3, 4
N, M = 5, 6

indices = torch.randint(0, N, (B, H, K))
x = torch.r…