When you wish to not update (freeze) parts of the network, the recommended solution is to set requires_grad = False, and/or (please confirm?) not send the parameters you wish to freeze to the optimizer input.
I would like to clarify that the requires_grad = False simply avoids unnecessary computation, update, and storage of gradients at those nodes and does not create subgraphs which saves memory.
However, the parameters with requires_grad = False will still contain a grad_fn (if it has one) so that in the backward pass, the gradient from that node is still technically still calculated passed backward and the chain rule is still maintained and used for the parameters with requires_grad = True?
So if we have layers L1 -> L2 -> LOSS and want to freeze L2, the gradient of LOSS wrt L1 requires the gradient of LOSS wrt L2 as per chain rule. My confusion is that as the name implies, requires_grad = False on L2 will not compute the gradient, but we need it to update L1, no?
Please confirm  Thank you