Reproducibility Questions
When initializing task-specific models from pre-trained language models, we see something like:

`Some weights of XLMRobertaForQuestionAnswering were not initialized from the model checkpoint at xlm-roberta-large and are ne
wly initialized: ['qa_outputs.weight', 'qa_outputs.bias']`

It seems that the initialization is model-dependant, however, I wonder whether this behaviour is controlled by the random seed that we set in the `training_args.py` `seed` hyper-parameter?

If not, I assume we should the following? But how? Are there any examples?

```
To ensure reproducibility across runs, use the: func:`~transformers.Trainer.model_init` function to instantiate the model if it has some randomly initialized parameters.
```

@sgugger 


If you are using any of the example scripts, they set the seed before instantiating the model for full reproducibility. If not, you should do that in your script by using the `set_seed` function you can import from the library, or by using the `model_init` function to initialize your model (the `Trainer` will set the seed before calling it).<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.