Thanks @Tony-Y for the informations! I didn’t know the PR exists and am surprised that it has been more than a year to verify. I knew that the Fast.ai’s solution but wanted to ask why the official PyTorch doesn’t have one since the AdamW is quite effective. I’ll follow up the PRs as well. Thank you again!