Hi, I am wondering when fine tune only part of parameters, what would happen if we only append parameters (to be trained) in the optimizer, but don’t set other parameters’ parameter.requires_grad = False.
Normally we would have:
model.feature1.requires_grad = False # Not training
model.feature2.requires_grad = True # to be trained

optimizer = Adam(model.feature2.parameters(), ...)

However, what if I do:
model.feature1.requires_grad = True # Not training
model.feature2.requires_grad = True # to be trained

optimizer = Adam(model.feature2.parameters(), ...)

Would this be ok?