Possible pickle issues
https://github.com/huggingface/transformers/blob/af2aac51fc1c59237ff7228908ace2cd8fc0d9a6/src/transformers/optimization.py#L49
When attempting to pickle this function, there is a potential for an error to occur due to the lambda function that cannot be pickled.
I suggest the following solution for this problem.


```
def get_constant_lambda(_):
    return 1

def get_constant_schedule(optimizer: Optimizer, last_epoch: int = -1):
    """
    Create a schedule with a constant learning rate, using the learning rate set in optimizer.

    Args:
        optimizer ([`~torch.optim.Optimizer`]):
            The optimizer for which to schedule the learning rate.
        last_epoch (`int`, *optional*, defaults to -1):
            The index of the last epoch when resuming training.

    Return:
        `torch.optim.lr_scheduler.LambdaLR` with the appropriate schedule.
    """

    return LambdaLR(optimizer, get_constant_lambda, last_epoch=last_epoch)
```




Would like to open a PR with your fix?<|||||>yes, is it okay?