you could use a backward hook on the output on fc2 to zero the gradients going through parts that you want to filter.
For example:
m = nn.Linear(1024, 4096)
input = Variable(torch.randn(128, 1024), requires_grad=True)

out = m(input)
def my_hook(grad):
    grad_clone = grad.clone()
    grad_clone[:, 2500:] = 0
    return grad_clone
h = out.register_hook(my_hook) # zeroes the gradients wrt the outputs for everything that's not 0 to 2500 over all mini-batches

out.backward(grads)

Edit: edited to incorporate @fmassa’s answer below