Hi.
I wanna implement network pruning using PyTorch.
I made a weight histogram to find out pruning point.
And then make weight, which can be pruned by histogram, zero.
At this point I have a few question.


Is there any method to make weights zero? I made a for 2~4 loop… Here is my code
 for m in net.modules():
 if isinstance(m, nn.Linear):
     wv_fc = m.weight.data

     if number_wv == 4:
         print("pruning FC1 weights")
         number = 0
         for n in wv_fc:
             for i in range(0, 100):
                 for j in range(0, 64*3*3):
                     if m.weight.data[i][j] <= 0.0404 and m.weight.data[i][j] >= -0.0404:
                         m.weight.data[i][j] = 0
                     print(m.weight.data[i][j])                             
             number = number + 1
         number_wv = number_wv + 1

     else:
         print("pruning FC2 weights")
         number = 0
         for n in wv_fc:
             for i in range(0, 10):
                 for j in range(0, 100):
                     if m.weight.data[i][j] <= 0.0404 and m.weight.data[i][j] >= -0.0404:
                         m.weight.data[i][j] = 0
                     print(m.weight.data[i][j])
             number = number + 1
         number_wv = number_wv + 1



If I done masking, is there any simple method to freeze the zero weight specifically? Not per layer like "reqires_grad=FALSE"
I wanna freeze only zero weights in entire network.
In this context, freeze means that freezed weights cannot be trained anymore.