Neural Network Algorithm, Neural Networks, Julialang, Editors Pick, Thoughts And Theory.
we redefined the forward step function to include the Hadamard product for the mask matrix : function (a:: PrunableDense)(x:: AbstractVecOrMat) W, b,sigma , M = a.dense.W, a.dense.b, a.dense.sigma , a.mask return sigma .((W.*M)*x .+ b) end Now you can use these prunable dense layers in the way you