Thanks for confirming that calling backward() on model1 will also free model2.
Your solutions are great but I want to keep model2 untouched because later I need to backward through its parameters, in order to achieve this goal, I tried retain_graph=True when calling backward() on model1, but this leads to memory explosion after many epochs.
How to keep model2 requires_grad without causing memory problem?