How to assign gpu when using run_language_modeling.py
## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.7.0
- Platform: Linux n188-182-130 4.14.81.bm.23-amd64
- Python version: 3.7.3
- PyTorch version (GPU?):1.7.1
- Tensorflow version (GPU?): 2.4.1
- Using GPU in script?:
- Using distributed or parallel set-up in script?:

### Who can help
- gpt2: @patrickvonplaten, @LysandreJik

## Information

Model I am using (gpt2.):

The tasks I am working on is:
* my own task or dataset: a txt file, each line is regarded as a sample because I set --line_by_line 


Dear all,

there are 8 GPU in my workspace, but from 0 to 3 have been occupied. So i can only use 4 GPUs from Rank4 to Rank7.
How could I set the parameters when using run_language_modeling.py? 

And could someone please explain it clearer, what the parameter --local_rank and --tpu_num_cores can do if I want to select only part of my GPU in this training.

Besides, i also found some bug in Trainer.
for example:
1. In initialization process, Trainer object has not attribute "prefiction_loss_only"  line 318
2. After training, when start doing evaluation, Trainer object has not attribute ‘is_world_master', I set --do_eval and give a eval set as input.
 

Here is the script I use in shell:

python3 run_language_modeling.py \
--output_dir $output \
--model_type 'gpt2' \
--model_name_or_path 'gpt2' \
--tokenizer_name 'bert-base-uncased' \
--cache_dir $pretrained_config \
--do_train true \
--train_data_file $train_file \
--do_eval true \
--eval_data_file $test_file \
--line_by_line true \
--mlm true \
--learning_rate 1e-4 \
--num_train_epochs 15 \
--per_device_train_batch_size 128 \


## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->

You should be able to control this with the `CUDA_VISIBLE_DEVICES` environment variable. See this [stackoverflow issue](https://stackoverflow.com/questions/39649102/how-do-i-select-which-gpu-to-run-a-job-on) for example<|||||>> You should be able to control this with the `CUDA_VISIBLE_DEVICES` environment variable. See this [stackoverflow issue](https://stackoverflow.com/questions/39649102/how-do-i-select-which-gpu-to-run-a-job-on) for example

Thank you for your fast reply.
I will try the solution mentioned in above.

By the way, when initializing Trainer in run_language_modeling.py, the errors Trainer object has not attribute "prediction_loss_only" and after training before evaluation Trainer object has not attribute ‘is_world_master' are still here, I just delete this two line, is this a bug or i miss some necessary parameters that should be input?<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.