Solved by ptrblck in post #2 
              
              
                A bit ugly, but for now this should work: 
a = torch.randn(4, 5, 100)
indices = torch.LongTensor([2, 4, 0, 1])
torch.gather(a, 1, indices.view(-1, 1).unsqueeze(2).repeat(1, 1, 100))