Using torch.gather could work. Yet, it will first generate a tensor with the same size as the original tensor. If any one have a better way?
def score_max(x, dim, score):
    _tmp=[1]*len(x.size())
    _tmp[dim] = x.size(dim)
    return torch.gather(x,dim,score.max(
        dim)[1].unsqueeze(dim).repeat(tuple(_tmp))).select(dim,0)