finetune_trainer.py script is not using given config file
## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.1.1 (stable)
- Platform: Google Colab
- Python version: 3.6.9
- PyTorch version (GPU?): 1.7.0+cu101
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: no

### Who can help

 Trainer: @sgugger
 examples/seq2seq: @patil-suraj

## Information

Model I am using (Bert, XLNet ...): Bart

The problem arises when using the official example scripts: `examples/seq2seq/finetune_trainer.py`

## Problem

When giving a local configuration file with `--config_name` the script first loads the config from the local files as expected, but then it loads a new configuration file from cache, which is not the one provided through the script's arguments:

```
2021-01-13 11:04:52.919133: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.10.1
01/13/2021 11:04:55 - WARNING - __main__ -   Process rank: -1, device: cuda:0, n_gpu: 1, distributed training: False, 16-bits training: True
01/13/2021 11:04:55 - INFO - __main__ -   Training/evaluation parameters Seq2SeqTrainingArguments(output_dir='/content/drive/My Drive/MAGMA: Summarization/fine-tuning/sshleifer?distilbart-cnn-12-6_karger_books_para_train', overwrite_output_dir=True, do_train=True, do_eval=True, do_predict=False, model_parallel=False, evaluation_strategy=<EvaluationStrategy.STEPS: 'steps'>, prediction_loss_only=False, per_device_train_batch_size=4, per_device_eval_batch_size=4, per_gpu_train_batch_size=None, per_gpu_eval_batch_size=None, gradient_accumulation_steps=1, eval_accumulation_steps=None, learning_rate=3e-05, weight_decay=0.0, adam_beta1=0.9, adam_beta2=0.999, adam_epsilon=1e-08, max_grad_norm=1.0, num_train_epochs=10.0, max_steps=-1, warmup_steps=0, logging_dir='runs/Jan13_11-04-55_a1d3ea40f4c6', logging_first_step=False, logging_steps=10, save_steps=1000, save_total_limit=3, no_cuda=False, seed=42, fp16=True, fp16_opt_level='O1', local_rank=-1, tpu_num_cores=None, tpu_metrics_debug=False, debug=False, dataloader_drop_last=False, eval_steps=1000, dataloader_num_workers=0, past_index=-1, run_name='/content/drive/My Drive/MAGMA: Summarization/fine-tuning/sshleifer?distilbart-cnn-12-6_karger_books_para_train', disable_tqdm=False, remove_unused_columns=True, label_names=None, load_best_model_at_end=False, metric_for_best_model='rougeL', greater_is_better='True', ignore_data_skip=False, fp16_backend='auto', sharded_ddp=False, label_smoothing=0.1, sortish_sampler=True, predict_with_generate=True, adafactor=True, encoder_layerdrop=None, decoder_layerdrop=None, dropout=None, attention_dropout=None, lr_scheduler='linear')
[INFO|configuration_utils.py:429] 2021-01-13 11:04:55,952 >> loading configuration file /content/drive/My Drive/MAGMA: Summarization/fine-tuning/sshleifer?distilbart-cnn-12-6_config/config.json
[INFO|configuration_utils.py:467] 2021-01-13 11:04:55,953 >> Model config BartConfig {
  "_num_labels": 3,
  "activation_dropout": 0.0,
  "activation_function": "gelu",
  "add_bias_logits": false,
  "add_final_layer_norm": false,
  "architectures": [
    "BartForConditionalGeneration"
  ],
  "attention_dropout": 0.0,
  "bos_token_id": 0,
  "classif_dropout": 0.0,
  "classifier_dropout": 0.0,
  "d_model": 1024,
  "decoder_attention_heads": 16,
  "decoder_ffn_dim": 4096,
  "decoder_layerdrop": 0.0,
  "decoder_layers": 6,
  "decoder_start_token_id": 2,
  "do_blenderbot_90_layernorm": false,
  "dropout": 0.1,
  "early_stopping": true,
  "encoder_attention_heads": 16,
  "encoder_ffn_dim": 4096,
  "encoder_layerdrop": 0.0,
  "encoder_layers": 12,
  "eos_token_id": 2,
  "extra_pos_embeddings": 2,
  "force_bos_token_to_be_generated": true,
  "id2label": {
    "0": "LABEL_0",
    "1": "LABEL_1",
    "2": "LABEL_2"
  },
  "init_std": 0.02,
  "is_encoder_decoder": true,
  "label2id": {
    "LABEL_0": 0,
    "LABEL_1": 1,
    "LABEL_2": 2
  },
  "max_length": 150,
  "max_position_embeddings": 1024,
  "min_length": 10,
  "model_type": "bart",
  "no_repeat_ngram_size": 5,
  "normalize_before": false,
  "normalize_embedding": true,
  "num_beams": 4,
  "num_hidden_layers": 12,
  "output_past": true,
  "pad_token_id": 1,
  "prefix": " ",
  "replacing_rate": 0,
  "scale_embedding": false,
  "static_position_embeddings": false,
  "student_decoder_layers": null,
  "student_encoder_layers": null,
  "task_specific_params": {
    "summarization": {
      "early_stopping": true,
      "length_penalty": 2.0,
      "max_length": 142,
      "min_length": 56,
      "no_repeat_ngram_size": 3,
      "num_beams": 4
    }
  },
  "use_cache": true,
  "vocab_size": 50264
}

01/13/2021 11:04:56 - INFO - filelock -   Lock 140709138970608 acquired on /content/drive/My Drive/MAGMA: Summarization/fine-tuning/sshleifer?distilbart-cnn-12-6_karger_books_para_train/adac95cf641be69365b3dd7fe00d4114b3c7c77fb0572931db31a92d4995053b.9307b6cec4435559ec6e79d5a210a334b17706465329e138f335649d14f27e78.lock
[INFO|file_utils.py:1301] 2021-01-13 11:04:56,231 >> https://huggingface.co/sshleifer/distilbart-cnn-12-6/resolve/main/config.json not found in cache or force_download set to True, downloading to /content/drive/My Drive/MAGMA: Summarization/fine-tuning/sshleifer?distilbart-cnn-12-6_karger_books_para_train/tmprjc9mj89
Downloading: 100% 1.62k/1.62k [00:00<00:00, 1.52MB/s]
[INFO|file_utils.py:1305] 2021-01-13 11:04:56,516 >> storing https://huggingface.co/sshleifer/distilbart-cnn-12-6/resolve/main/config.json in cache at /content/drive/My Drive/MAGMA: Summarization/fine-tuning/sshleifer?distilbart-cnn-12-6_karger_books_para_train/adac95cf641be69365b3dd7fe00d4114b3c7c77fb0572931db31a92d4995053b.9307b6cec4435559ec6e79d5a210a334b17706465329e138f335649d14f27e78
[INFO|file_utils.py:1308] 2021-01-13 11:04:56,518 >> creating metadata file for /content/drive/My Drive/MAGMA: Summarization/fine-tuning/sshleifer?distilbart-cnn-12-6_karger_books_para_train/adac95cf641be69365b3dd7fe00d4114b3c7c77fb0572931db31a92d4995053b.9307b6cec4435559ec6e79d5a210a334b17706465329e138f335649d14f27e78
01/13/2021 11:04:56 - INFO - filelock -   Lock 140709138970608 released on /content/drive/My Drive/MAGMA: Summarization/fine-tuning/sshleifer?distilbart-cnn-12-6_karger_books_para_train/adac95cf641be69365b3dd7fe00d4114b3c7c77fb0572931db31a92d4995053b.9307b6cec4435559ec6e79d5a210a334b17706465329e138f335649d14f27e78.lock
[INFO|configuration_utils.py:431] 2021-01-13 11:04:56,522 >> loading configuration file https://huggingface.co/sshleifer/distilbart-cnn-12-6/resolve/main/config.json from cache at /content/drive/My Drive/MAGMA: Summarization/fine-tuning/sshleifer?distilbart-cnn-12-6_karger_books_para_train/adac95cf641be69365b3dd7fe00d4114b3c7c77fb0572931db31a92d4995053b.9307b6cec4435559ec6e79d5a210a334b17706465329e138f335649d14f27e78
[INFO|configuration_utils.py:467] 2021-01-13 11:04:56,523 >> Model config BartConfig {
  "_num_labels": 3,
  "activation_dropout": 0.0,
  "activation_function": "gelu",
  "add_bias_logits": false,
  "add_final_layer_norm": false,
  "architectures": [
    "BartForConditionalGeneration"
  ],
  "attention_dropout": 0.0,
  "bos_token_id": 0,
  "classif_dropout": 0.0,
  "classifier_dropout": 0.0,
  "d_model": 1024,
  "decoder_attention_heads": 16,
  "decoder_ffn_dim": 4096,
  "decoder_layerdrop": 0.0,
  "decoder_layers": 6,
  "decoder_start_token_id": 2,
  "do_blenderbot_90_layernorm": false,
  "dropout": 0.1,
  "early_stopping": true,
  "encoder_attention_heads": 16,
  "encoder_ffn_dim": 4096,
  "encoder_layerdrop": 0.0,
  "encoder_layers": 12,
  "eos_token_id": 2,
  "extra_pos_embeddings": 2,
  "force_bos_token_to_be_generated": true,
  "id2label": {
    "0": "LABEL_0",
    "1": "LABEL_1",
    "2": "LABEL_2"
  },
  "init_std": 0.02,
  "is_encoder_decoder": true,
  "label2id": {
    "LABEL_0": 0,
    "LABEL_1": 1,
    "LABEL_2": 2
  },
  "length_penalty": 2.0,
  "max_length": 142,
  "max_position_embeddings": 1024,
  "min_length": 56,
  "model_type": "bart",
  "no_repeat_ngram_size": 3,
  "normalize_before": false,
  "normalize_embedding": true,
  "num_beams": 4,
  "num_hidden_layers": 12,
  "output_past": true,
  "pad_token_id": 1,
  "prefix": " ",
  "replacing_rate": 0,
  "scale_embedding": false,
  "static_position_embeddings": false,
  "student_decoder_layers": null,
  "student_encoder_layers": null,
  "task_specific_params": {
    "summarization": {
      "early_stopping": true,
      "length_penalty": 2.0,
      "max_length": 142,
      "min_length": 56,
      "no_repeat_ngram_size": 3,
      "num_beams": 4
    }
  },
  "use_cache": true,
  "vocab_size": 50264
}
```
You can see for example that the `min_length` parameter is different in the second output, which is the default one and not the one provided by me.
Hi @marcoabrate 

the script uses the provided `config` file, the reason you see `max_length` 56 because the script replaces the generate params (max/min_len etc) in `config` using `task_specific_params` 

https://github.com/huggingface/transformers/blob/245cdb469d2a7f47316926fdbac925e0ed149332/examples/seq2seq/finetune_trainer.py#L216

Here in the `config`, `min_length` is 56  in `task_specific_params` so 10 get's changed to 50. <|||||>Thank you. I have managed to make it work with my configuration parameters, the problem was indeed the task specific params.
In any case, I think the `loading configuration file https://huggingface.co/sshleifer/distilbart-cnn-12-6/resolve/main/config.json from cache` message and the second config print are a bit misleading.<|||||>This issue has been automatically marked as stale and been closed because it has not had recent activity. Thank you for your contributions.

If you think this still needs to be addressed please comment on this thread.