Hello.
I want to exclude some parameters in the optimizer during training.
Specifically, i want to freeze the updates of some layers during training.
(however, requires_grad=False is not working.)
Before starting the training, i construct an optimizer
optimizer = torch.optim.SGD(net.parameters(), lr, momentum=momentum, weight_decay=decay, nesterov=True)

scheduler = torch.optim.lr_scheduler.LambdaLR(
    optimizer,
    lr_lambda=lambda step: cosine_annealing(
        step,
        schedule_epoch * len(train_loader),
        1,  # since lr_lambda computes multiplicative factor
        1e-6 / 0.1))

for epoch in range(total_epoch):
  ...
  # update the models

In this scenario. i want to freeze some layers (exactly, front layers) after some epochs.
How can i exclude the updated parameters in above optimizer?
Thank you.