Oh, in that case, check out the Per-parameter options here 2.
Essentially, you need to create different parameter groups with different hyperparameters and pass them as a dict to the optimizer instead of model.parameters()
For instance,
opt = optim.SGD([{'params': layer_1.parameters(), 'lr': 0.01}, {'params': layer_2.parameters(), 'lr': 0.01, 'weight_decay': 0.01}])

I hope that helps.