Fine tune with local model raised `torch.nn.modules.module.ModuleAttributeError: 'DataParallel' object has no attribute 'config'`
# ❓ Questions & Help

I've fined tune distilgpt2 with `run_language_modeling.py` under my local `output_dir` and want to fine tune with the model in `output_dir` again, but it raised `torch.nn.modules.module.ModuleAttributeError: 'DataParallel' object has no attribute 'config'`. 

## Details
<!-- Description of your issue -->
I've fined tune distilgpt2 with `run_language_modeling.py` as follows:
```shell
python3 run_language_modeling.py     --output_dir=/home/xxx/gpt_model/transformers/examples/language-modeling/output_dir --model_type=gpt2  --model_name_or_path=distilgpt2 --per_device_train_batch_size=10     --do_train     --train_data_file=/home/xxx/gpt_model/data_info/data.txt --block_size=64 --save_steps=100 --overwrite_output_dir
```
It works fine and i can get the fine-tuned model after training with my data `data.txt`. Now i want to fine-tune in the `output_dir` so i run `run_language_modeling.py` as follows on my `new_data.txt`:
```shell
 python3 run_language_modeling.py     --output_dir=/home/xxx/gpt_model/transformers/examples/language-modeling/output_new_data --model_type=gpt2  --model_name_or_path=/home/xxx/gpt_model/transformers/examples/language-modeling/output_dir/  --per_device_train_batch_size=20  --do_train     --train_data_file=/home/xxx/gpt_model/data_info/new_data.txt --block_size=64 --save_steps=1000 --overwrite_output_dir
```
But it raise exception and exit. The stderr is as follows
```shell
/home/xxx/gpt_model/pytorch/pytorch/torch/optim/lr_scheduler.py:235: UserWarning: Please also save or load the state of the optimizer when saving or loading the scheduler.
  warnings.warn(SAVE_STATE_WARNING, UserWarning)
Traceback (most recent call last):
  File "run_language_modeling.py", line 320, in <module>
    main()
  File "run_language_modeling.py", line 284, in main
    trainer.train(model_path=model_path)
  File "/home/xxx/anaconda3/envs/transformers/lib/python3.6/site-packages/transformers/trainer.py", line 683, in train
    self.total_flos = getattr(model.config, "total_flos", 0)
  File "/home/xxx/gpt_model/pytorch/pytorch/torch/nn/modules/module.py", line 779, in __getattr__
    type(self).__name__, name))
torch.nn.modules.module.ModuleAttributeError: 'DataParallel' object has no attribute 'config'
```
##### (1) I'm training on multiple GPUs, but i did'n specify a specific gpu to run. So at first i think this may caused by multiple GPUs, and add the following code under https://github.com/huggingface/transformers/blob/52d250f6aa14844024806e5e4dd1c7882bbd8dd5/src/transformers/trainer.py#L641
```shell
if isinstance(model,torch.nn.DataParallel):
		model = model.module
```
It has no error but doesn't work. The output is as follows . After that, the process is killed and return code `echo $?` is `0`.
```shell
Epoch: 0it [00:00, ?it/s]
/home/lenajin/anaconda3/envs/transformers/lib/python3.6/site-packages/transformers/trainer.py:1087: FutureWarning: This method is deprecated, use `Trainer.is_world_process_zero()` instead.
  warnings.warn("This method is deprecated, use `Trainer.is_world_process_zero()` instead.", FutureWarning)
```
##### (2) Following with https://github.com/huggingface/transformers/issues/1991 , I try to train with following script to train with gpu whose id is `1`.
```shell
CUDA_VISIBLE_DEVICES=1 python3 run_language_modeling.py     --output_dir=/home/xxx/gpt_model/transformers/examples/language-modeling/output_dir --model_type=gpt2  --model_name_or_path=distilgpt2 --per_device_train_batch_size=20     --do_train     --train_data_file=/home/xxx/gpt_model/data_info/data.txt --block_size=64 --save_steps=100 --overwrite_output_dir
```
it return with return code `echo $?` =`0`. At first i don't know what's the problem, but now i changed `--per_device_train_batch_size=20    ` from 20 to 10. It seems all is well, and now my `GPU-Util` is about `98%`.  Maybe it return without training duing to my limit gpu CC？It's wired but at least it works now. Maybe there can give some error message about the return reason?

I haven't used the trainer but I think maybe you just need to change the line `self.total_flos = getattr(model.config, "total_flos", 0)` to 

`_model = model.module if hasattr(model, 'module') else model `
`self.total_flos = getattr(_model.config, "total_flos", 0)`<|||||>This is a breaking change that was not announced - it broke our production script.<|||||>@sgugger might be interested in this.

@marrrcin what is the breaking change? This seems to be a bug rather than an intentional change.<|||||>I meat it's a bug that broke our usage of Trainer in production - we're using single-multigpu-machine to train our models and https://github.com/huggingface/transformers/blob/9e68d075a4100906509170498480823e7e61874a/src/transformers/trainer.py#L698 basically breaks it after upgrade to `3.2.0`.
Maybe this is enough for the fix: 
https://github.com/huggingface/transformers/pull/7384
<|||||>I see! Thanks for opening a PR!<|||||>My bad, I think this bug slipped through with one of the style changes at code review on one of my PRs. Thanks for raising the issue!<|||||>Do you think that it will be possible to move this fix forward and release patch over the weekend? We don't want to have custom package installation in our CI.<|||||>We're having a new release (v3.3.0) on Monday, is that soon enough? The fix will be in it.<|||||>That would be great!<|||||>I am facing the same issue. Whenever I continues my training from certain checkpoint instead of training from scratch. I got the exact same error as OP. Should I just wait for the new release on tomorrow?<|||||>Hi,
I tested this PR, this does not fully solve the issue, I am getting these error during evaluation of seq2seq_trainer.


 File "finetune_t5_trainer.py", line 233, in <module>
    main()
  File "finetune_t5_trainer.py", line 188, in main
    result = trainer.evaluate(eval_datasets, compute_metrics_fn)
  File "/home/rabeeh/internship/seq2seq/t5_trainer.py", line 175, in evaluate
    prediction_loss_only=True if self.compute_metrics is None else None, # self.compute_metrics[eval_task]
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/transformers/trainer.py", line 1452, in prediction_loop
    metrics = self.compute_metrics(EvalPrediction(predictions=preds, label_ids=label_ids))
  File "/home/rabeeh/internship/seq2seq/metrics/metrics.py", line 112, in translation_metrics
    pred_str, label_str = decode_pred(pred)
  File "/home/rabeeh/internship/seq2seq/metrics/metrics.py", line 99, in decode_pred
    label_str = tokenizer.batch_decode(pred.label_ids, skip_special_tokens=True)
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/transformers/tokenization_utils_base.py", line 2966, in batch_decode
    for seq in sequences
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/transformers/tokenization_utils_base.py", line 2966, in <listcomp>
    for seq in sequences
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/transformers/tokenization_utils_base.py", line 3002, in decode
    **kwargs,
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/transformers/tokenization_utils.py", line 732, in _decode
    filtered_tokens = self.convert_ids_to_tokens(token_ids, skip_special_tokens=skip_special_tokens)
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/transformers/tokenization_utils.py", line 716, in convert_ids_to_tokens
    tokens.append(self._convert_id_to_token(index))
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/transformers/tokenization_t5.py", line 245, in _convert_id_to_token
    token = self.sp_model.IdToPiece(index)
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/sentencepiece/__init__.py", line 501, in _batched_func
    return _func(self, arg)
  File "/opt/conda/envs/internship/lib/python3.7/site-packages/sentencepiece/__init__.py", line 494, in _func
    raise IndexError('piece id is out of range.')
IndexError: piece id is out of range.
42it [01:31,  2.18s/it]

<|||||>@rabeehk this seems like a completely different issue. Please open a new issue.