Can't pickle local object using the finetuning example.
I was testing out the finetuning example from the repo:

`python run_lm_finetuning.py --train_data_file="finetune-output/KantText.txt" --output_dir="finetune-output/hugkant" --model_type=gpt2 --model_name_or_path=gpt2 --do_train --block_size=128`

While saving the checkpoint, it gives the following error:

```

Traceback (most recent call last):
  File "run_lm_finetuning.py", line 790, in <module>
    main()
  File "run_lm_finetuning.py", line 740, in main
    global_step, tr_loss = train(args, train_dataset, model, tokenizer)
  File "run_lm_finetuning.py", line 398, in train
    torch.save(scheduler.state_dict(), os.path.join(output_dir, "scheduler.pt"))
  File "D:\Software\Python\lib\site-packages\torch\serialization.py", line 209, in save
    return _with_file_like(f, "wb", lambda f: _save(obj, f, pickle_module, pickle_protocol))
  File "D:\Software\Python\lib\site-packages\torch\serialization.py", line 134, in _with_file_like
    return body(f)
  File "D:\Software\Python\lib\site-packages\torch\serialization.py", line 209, in <lambda>
    return _with_file_like(f, "wb", lambda f: _save(obj, f, pickle_module, pickle_protocol))
  File "D:\Software\Python\lib\site-packages\torch\serialization.py", line 282, in _save
    pickler.dump(obj)
AttributeError: Can't pickle local object 'get_linear_schedule_with_warmup.<locals>.lr_lambda'
```
Do you mind specifying which versions of everything you're using, as detailed in the [bug report issue template](https://github.com/huggingface/transformers/issues/new/choose)?<|||||>Hi @Normand-1024 

were you able to fix this error?
as i am getting the same error while trying to run glue task (QQP) but works fine when i run MRPC.<|||||>Hi,

I was able to get rid of this error by upgrading the torch version.<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
<|||||>Still having this issue with transformers `2.11.0`.<|||||>@lucadiliello did you maange to fix it?<|||||>Not yet. A solution would be to use `dill` instead of `pickle`... but I'm not sure how to do it.<|||||>Getting same error,
Not sure how to fix this error.<|||||>same error, with all newest version<|||||>I solved by reimplementing all the schedulers without lambda functions. [here](https://github.com/iKernels/transformers-lightning) I published many schedulers.<|||||>Same error with all newest version too.

![image](https://user-images.githubusercontent.com/44800122/115481664-afdd6300-a27f-11eb-9a6e-77b5ed4afe02.png)
<|||||>UP. 
Is it a package version related issue?<|||||>Having the same issue: 
`Can't pickle local object 'get_linear_schedule_with_warmup.<locals>.lr_lambda'`<|||||>For running the example scripts passing `--no_multi_process` solved it for me.

I haven't looked into the huggingface code yet but I could imagine that [this](https://stackoverflow.com/questions/52265120/python-multiprocessing-pool-attributeerror) is the bug here. I think it only shows up when `spawn` instead of `fork` is used to create new processes, which is why the developers might have missed it.<|||||>I set the `gpus=1`, and it works. <|||||>Well, this seems that it is a local object that can not be forked, you may define it at each forked process. This may work well. However, somebody should fix it.