Right! I know what you mean, I just want to make the convolution time of time1 and time2 to be identical, then I can reduce the addtional time cost brounght by the
temp_in = torch.index_select(test_input2, 1, Variable(k_in_mask))

and
output.index_copy(1, Variable(k_out_mask), temp_out)

Maybe this is meaningless， because I just want to use the index_select, small convolution and index_copy to replace large convolution, but I don’t find that the small convolution brings additional noise time cost.
Maybe the operation is not so meaningful