Say I have two DNN modules F and G.
out = F(G(x_1), x_2)
And a loss function like loss(out, label).
I want to disable gradients for F during one particular update (though not necessarily in others).
I tried this during that update:
g = G(x_1)
with torch.no_grad():
   out = F(g, x_2)

But (correct me if I’m wrong), it seems gradients don’t propagate back to G in this case. I need gradients to still propagate back to G.