I have a network N1 with some layers and a network N2. They have some layers that are different. They’re trained on different data. I want to get gradients for the common layers, say, l1, l2, l5 and add them and continue the training process.
I was thinking of getting gradients from for name, parameter in model.named_parameters(): and filtering by name and adding them and calling optimizer.step(). But, the model.named_parameters is read only. How do I access the gradients and perform operations on them?