Hi @albanD, I am currently using PyTorch v1.9.0.
Could you please tell me why is that the case if the optimizer involves weight decay, then the freezed parameters would still change? I am facing exact same problem, where part of the network is supposed to be freezed (say the encoder), and the rest of the network has to be trained, but with weight decay ( I am using 0.01 in my case) with Adam.
To do so, before starting the training I just did -
for p in model.feat.parameters():
	p.requires_grad=False

where I intend to freeze the parameters of model.feat, and then I instantiate the optimizer as follows:
optim = torch.optim.Adam(model.parameters(), lr = args['lr'], weight_decay=args['weight_decay'])

Thanks!!