Thanks @albanD so let’s say if I do
optimizer = Adam(model.parameters(), lr=0.1)
for batch in batch_generator:
    model.layer1.requires_grad=False
    loss = model(batch)
    loss.backward()
    optimizer.step()

the params in layer1 won’t be updated because of requires_grad=False?
Do I need to explicitly set model.layer1.grad=None? In my case, without layer1.grad=None, the weights are still updated somehow although the values in model.layer1.grad are zeros.