If you really just want the indices [0, 1, 2], a.diag() will give you what you want.
If the indices can be other things than the “diagonal” elements, a.gather(1, b.unsqueeze(1)) will give you what you want.