Learning rate is set to zero for the entirety of the first epoch
https://github.com/huggingface/transformers/blob/31d452c68b34c2567b62924ee0df40a83cbc52d5/src/transformers/optimization.py#L210

Both in practice and based on my understanding of the code, this will produce a `LambdaLR` that will return a multiplicative factor of 0 for the entirety of the first epoch (as `current_step` will be 0), which will mean that the entire first epoch the model will not train.

The names given to variables in the code imply this might be intended to be set with training steps, not epochs; is that the desire? If not, should this be modified to account for the first epoch having `current_step` equal to 0? Or is something wrong in my specific use-case?

My use-case is in using the `get_polynomial_decay_schedule_with_warmup` as a scheduler in a pytorch lightning module. Note I also mentioned this on the forum, here: https://discuss.huggingface.co/t/huggingface-lr-decay-schedulers-spend-the-first-epoch-w-an-lr-of-0/28195
Diving deeper into this, I think this is due to a mismatch between pytorch lightning's design and huggingface's. Huggingface trainer. [Huggingface's trainer calls the lr scheduler every step](https://github.com/huggingface/transformers/blob/31d452c68b34c2567b62924ee0df40a83cbc52d5/src/transformers/trainer.py#L1845), and [pytorch lightning can be configured to either call it once per step or once per epoch](https://github.com/Lightning-AI/lightning/blob/612d43e5bf38ba73b4f372d64594c2f9a32e6d6a/src/pytorch_lightning/loops/epoch/training_epoch_loop.py#L407), so I likely have something configured wrong.