Hi,
You can very easily set the gradient for this entry to 0 at every backward pass by simply adding a hook on the weights like:
def hook_fn(grad):
    res = grad.clone() # You are not allowed to change the input
    res[1][1] = 0 # assumes 3x3 kernel
model.cnn1.weight.register_hook(hook_fn) # For the first layer of your model above.

The limitation of this approach is that optimizers like Adam or SGD with momentum will update parameters even if the gradient is exactely 0.
Another approach can be to manually set it back to 0 before the forward
def pre_fw_hook(mod, *args):
    with torch.no_grad(): # Don't track this in gradient computations
        mod.weight[1][1] = 0 # Assumes 3x3 kernel
model.cnn1.register_pre_forward_hook(pre_fw_hook)

But then the weights you read after the weight update might not contain the 0 and you need to be careful to register this hook backward again after loading the model.