If you want so set some filters as zeros, you could either remove them from your model completely or alternatively multiply your filters with a mask containing 1s for the valid filters and 0s for the filters you want to zero out, and apply the same mask to the gradients of these filters.