I am trying to understand how to get the “freeze” weights functionality work. Suppose I have a multi-layer network:  x --> L1 --> L2 --> L3 --> y .  (where x is input and y is output)
Suppose I want to freeze L2 layer in pytorch (only the L2, keeping L1 and L3 trainable).
As discussed in [1] and bunch other posts, I simply set requires_grad=False for all params in L2, which disables gradient computation for L2 params. The observation there is, they are making layers close to the loss_function as trainable and layers away from loss_func are frozen (i.e. all trainable params are connected to the loss in the gradient graph,)
In my case, if I freeze L2,  does L1 still get trained in the back prop? If yes, how?
[1] How the pytorch freeze network in some layers, only the rest of the training? 58