I would go with option1 , freeze what you don’t wanna update with this
for param in child.parameters():
    param.requires_grad = False

and then
optimizer = torch.optim.Adam(filter(lambda p: p.requires_grad, whole_model.parameters()), lr=opt.lr, amsgrad=True)