In general the default of all optimizers for weight decay is 0 (I don’t know why pytorch set 0.01 for just AdamW, all other optimizers have a default at 0) because you have to opt-in for weight decay. Even if it’s true that Adam and AdamW behave the same way when the weight decay is set to 0, I don’t think it’s enough to change that default behavior (0.01 is a great default otherwise, that is the one we set in fastai for the Learner after countless experiments, but I think it should be set in a higher-level API, not the optimizer itself).
And like @BramVanroy said, it would be such a breaking change that even if we really wanted to change that default, we probably wouldn’t.