Would you like to keep the “unlearned kernel” in its original state and use the gradients to update a clone of this kernel?
If so, note that the clone of this kernel would eventually get gradients for completely different parameters.
I’m unsure if I understood the question correctly, so please feel free to add more information about the use case.