After playing a bit with gather, I got something that almost gives me what I need:
result = a.gather(1, labs.unsqueeze(1))

result has shape (4, 1) instead of (4) though. Obviously this can be solved by using torch.squeeze function, however I am interested to know if there is a more efficient version that achieves this.