Two different solutions you can try.

You can specify to not process the gradient on a Variable with : variable.requires_grad = False
Then use your optimizer as:

optimizer = torch.optim.Adam(filter(lambda p: p.requires_grad, model.parameters()), learning_rate)


Process the gradient on all your Variables and choose which one you want to update with your optimizer.

optim.SGD([
                {'params': model.base.parameters()},
                {'params': model.classifier.parameters(), 'lr': 1e-3}
            ], lr=1e-2, momentum=0.9)