Solved by phan_phan in post #7 
              
              
                That is why (in the diagonal case), you can create a parameter w with only the size: 
[out_channels, in_channels, kernel_size] (for example : [16, 32, 3]). 
This parameter will only contain the useful diagonal information, without the useless zeros. So at the optimization step, only these floats wil…