you save the filters like this
class CustomConv(nn.module):
    def __init__(self):
        filters = nn.Parameter(# filter tensor)

so autograd registers the filter tensor as a parameter that needs to be optimized.
And you should return the activation map in the forward method like this
    def forward(self, input):
    # custom convolutions
    return activation_map