Even though I agree about the default value (it should probably be 0.01 as in the PyTorch 3 implementation), this probably should not be changed without warning because it breaks backwards compatibility. It was also implemented in transformers before it was available in PyTorch itself.
I guess it is implemented in this way, because most of the time you decide in the initialization which parameters you want to decay and which ones shouldn’t be decayed, such as here:

  
      github.com
  
  
    huggingface/transformers/blob/a75c64d80c76c3dc71f735d9197a4a601847e0cd/examples/contrib/run_openai_gpt.py#L230-L237 14

optimizer_grouped_parameters = [
    {
        "params": [p for n, p in param_optimizer if not any(nd in n for nd in no_decay)],
        "weight_decay": args.weight_decay,
    },
    {"params": [p for n, p in param_optimizer if any(nd in n for nd in no_decay)], "weight_decay": 0.0},
]
optimizer = AdamW(optimizer_grouped_parameters, lr=args.learning_rate, eps=args.adam_epsilon)