Hi KFrank!
Thank you very much for your reply，I tried what you said
params += [{'params': self.awl.parameters(), 'lr': lr, 'weight_decay': 0}]

But the error happened :
ValueError: some parameters appear in more than one parameter group

Then I tried to print params， here’s what print in the last part of this dict:
{'params': [Parameter containing:
tensor([1., 1.], requires_grad=True)], 'lr': 0.001, 'weight_decay': 0.0005},
 {'params': <generator object Module.parameters at 0x000001F771AEFBA0>, 'lr': 0.001, 'weight_decay': 0}]

That means I get an error if I add, and an error if I don’t 
But if I don’t add loss.params the code will be fine…
Best.
tianle