How to initialize AdamW optimizer in HuggingFace transformers?
Hello,
I am new to Python and NLP and so I have some questions that may sound a bit funny to the experts.

I had been trying to set my optimizer by setting
`optimizer = AdamW()`

but of course it failed, because I did not specify the required parameter `'param'` (for lr, betas, eps, weight_decay, and correct_bias, I am just going to use the default values).

As a beginner, I am not so clear on what `'param'` stands for in this case. What kind of input should I provide for `'param'`?

Thank you,

I had the same issue, apparently it should be model.params()<|||||>Thank you! This is helpful<|||||>You have to tell the optimizer which parameters it should optimize. Theoretically you could use multiple optimizers for different parameters. This is useful if you want to use different learning rates or different weight decays.

If your question is answered, please close the question. <|||||>> You have to tell the optimizer which parameters it should optimize. Theoretically you could use multiple optimizers for different parameters. This is useful if you want to use different learning rates or different weight decays.
> 
> If your question is answered, please close the question.

I'm guessing this may have something to do with how the params are set in this code from the squad example

```
    # Prepare optimizer and schedule (linear warmup and decay)
    no_decay = ["bias", "LayerNorm.weight"]
    optimizer_grouped_parameters = [
        {
            "params": [p for n, p in model.named_parameters() if not any(nd in n for nd in no_decay)],
            "weight_decay": args.weight_decay,
        },
        {"params": [p for n, p in model.named_parameters() if any(nd in n for nd in no_decay)], "weight_decay": 0.0},
    ]
    optimizer = AdamW(optimizer_grouped_parameters, lr=args.learning_rate, eps=args.adam_epsilon)
    scheduler = get_linear_schedule_with_warmup(
        optimizer, num_warmup_steps=args.warmup_steps, num_training_steps=t_total
    )
```

https://github.com/huggingface/transformers/blob/master/examples/run_squad.py

