Does the default weight_decay of 0.0 in transformers.AdamW make sense?
# ❓ Questions & Help

## Details
Hi,

I tried to ask in SO before, but apparently the question seems to be irrelevant. Anyways, here it is:

In the [Docs](https://huggingface.co/transformers/main_classes/optimizer_schedules.html#transformers.AdamW) we can clearly see that the AdamW optimizer sets the default weight decay to 0.0.

Given that the whole purpose of AdamW is to decouple the weight decay regularization, is my understanding that the results anyone can get with AdamW and Adam if both are used with `weight_decay=0.0` (this is, without weight decay) should be exactly the same.

Therefore, shouldn't make more sense to have the default weight decay for AdamW > 0?

Thank you so much!
Too bad you didn't get an answer on SO. I think you would multiply your chances of getting a good answer if you asked it over at https://discuss.huggingface.co!<|||||>Sure, I will ask there. Thank you!<|||||>I use weight decay and not use weight and surprisingly find that they are the same, why?