mode_val = weights.mode(dim=0)[0]
mode_val = mode_val.unsqueeze(0).repeat_interleave(10, dim=0)
inds = (weights != mode_val).float().argmax(dim=0)
Now, I just need to get the values of given indices for each column.
I am trying with index_select, gather and scatter_ but no luck so far.
Basically, I need something that takes a vector of indices and then
select those indices from each column in a matrix.
Note: Without the argmax, there is also a Boolean matrix of same shape to select.
I couldn’t use that to get the values and thus I was trying with indices of each column.

I have also tried masked_select with Boolean matrix but that doesn’t seem to do it.