Thank you for your response. But I did modify the code to avoid inplace operation like below
trained_model[i][0][k]= trained_model[i][0][k].add(intialized_model[i][0][k]) [In second else]
trained.add(intialized)  [In final else]

But it still showing the same error.
I didn’t understand one of you statement “I don’t think you want the gradients to flow back to network 2 right?” == I am just using Network 2 only for my copying purpose.
My goal is train Network after copying zero weights and freezing non-zero weight.
can you help me with an example of on how to torch.no_grad() in this scenario. I don’t know how to set it for a single tensor