Solved by ptrblck in post #2 
              
              
                You could use the gather operation and filter out the specific values afterwards: 
logits = torch.randn(2, 3, 4, 4)
target = torch.randint(0, 3, (2, 4, 4))
res = logits.gather(1, target.unsqueeze(1))