Hi,
Probably I am getting you or @t.g wrong, but is it possible that what you both say are different?
As I understand from what you say, we can set requires_grad=False for L2 in order to avoid training its parameters while we keep L1’s and L3’s parameters trainable.
From @t.g I understand that we cannot set requires_grad=False for L2 and instead we need to exclude it from the optimizer.
So which is the proper way to not update a specific layer while updating all the others? Preferably without excluding it from the optimizer?