I use the SGD optimiazer. For leraning rate, when using ‘requires_grad=False’, i ignore previous layers and set lr of last layer to be 0.01. When using ‘lr=0’, i set lrs of previous layers to be 0 and last layer to be 0.01.
According to my knowledge, ‘lr=0’ should not update parameters and thus equals to 'requires_grad=False '.(Though it maight lead to more computation.)