Solved by zcajiayin in post #3 
              
              
                Just find a way to do this using gather: 
dummy = B.unsqueeze(2).expand(B.size(0), B.size(1), A.size(2))
out = torch.gather(A, 1, dummy)