I have found a solution.
My problem is to start the gradient process from a specific filter in a specific layer with respect to the input image.
First, I have assigned a forward hook to all conv. layers in order to keep output of each filter (activation map). Second, I have assigned a backward hook to all layers to keep their gradient in the backward pass.
Third, I fed the model with an input image to obtain all activation maps. Then, I called the backward function on the norm of one of the activation maps (remind that each activation map of each filter is similar to the output of loss). Finally, I have all gradients in a list variable where the last element of this list is the gradient of input image which should have the size of (1,3,224,224) for instance.