I meant you could add extra tensor into optimizer by optimizer.add_param_group
or initialize your optimizer as follows:
optim.SGD([
                {'params': model.base.parameters()},
                {'params': model.classifier.parameters(), 'lr': 1e-3}
            ], lr=1e-2, momentum=0.9)