Thanks for your reply and I have another questions. In the forward, the filter does collect the directional information. But the backward still compute the gradient which change the zeros in original filter. This solution cannot keep the shape of filter?