I am asking if its “necessary” to do:-
for param in res_mod.parameters():
    param.requires_grad = False

if I want to “freeze” a part of a model. According to what I know, if we do this:-
optimizer_ft = optim.SGD(res_mod.fc.parameters(), lr=0.001, momentum=0.9)

then also we are only updating the weights of the ‘FC’ layer(or any other layer for that matter) and not the rest of the model. But yes, I agree it will be computationally expensive. I am asking this just to check if my understanding of how the optimizers in PyTorch work is correct or not.