ValueError while running run_glue.py with xlnet model.
## Environment info
+ transformers version: 3.5.1
+ Platform: Linux version 3.10.107-1-tlinux2-0050 
+ Python version: 3.7.6
+ PyTorch version (GPU?): 1.6.0+cu101
+ Tensorflow version (GPU?): no
+ Using GPU in script?: yes
+ Using distributed or parallel set-up in script?: yes

## Who can help
@sgugger 

## To reproduce
```shell
CUDA_VISIBLE_DEVICES=1,2 python run_glue.py \
  --model_name_or_path xlnet-base-cased \
  --task_name stsb \
  --do_eval \
  --max_seq_length 128 \
  --per_device_train_batch_size 32 \
  --per_device_eval_batch_size 8 \
  --max_steps 1200 \
  --learning_rate 2e-5 \
  --num_train_epochs 3.0 \
  --output_dir ./output/tranformer/xlnet \
  --cache_dir ./pretained_model/xlnet \
  --overwrite_output_dir \
  --overwrite_cache \
  --eval_accumulation_steps 2 \
  --gradient_accumulation_steps 1 \
  --disable_tqdm True\
  --dataloader_drop_last \
  --past_index 2
```
##  error
```shell
[INFO|trainer.py:1387] 2020-11-18 15:21:21,084 >> ***** Running Evaluation *****
[INFO|trainer.py:1388] 2020-11-18 15:21:21,084 >>   Num examples = 4000
[INFO|trainer.py:1389] 2020-11-18 15:21:21,085 >>   Batch size = 16
./sim/lib/python3.7/site-packages/transformers/modeling_xlnet.py:297: UserWarning: Mixed memory format inputs detected while calling the operator. The operator will output contiguous tensor even if some of the inputs are in channels_last format. (Triggered internally at  /pytorch/aten/src/ATen/native/TensorIterator.cpp:918.)
  attn_score = (ac + bd + ef) * self.scale
./sim/lib/python3.7/site-packages/torch/nn/parallel/_functions.py:61: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.
  warnings.warn('Was asked to gather along dimension 0, but all '
Traceback (most recent call last):
  File "run_glue.py", line 414, in <module>
    main()
  File "run_glue.py", line 366, in main
    eval_result = trainer.evaluate(eval_dataset=eval_dataset)
  File "./sim/lib/python3.7/site-packages/transformers/trainer.py", line 1313, in evaluate
    prediction_loss_only=True if self.compute_metrics is None else None,
  File "./sim/lib/python3.7/site-packages/transformers/trainer.py", line 1431, in prediction_loop
    preds_gatherer.add_arrays(self._gather_and_numpify(preds_host, "eval_preds"))
  File "./sim/lib/python3.7/site-packages/transformers/trainer_pt_utils.py", line 330, in add_arrays
    slice_len = self._nested_set_tensors(self._storage, arrays)
  File "./sim/lib/python3.7/site-packages/transformers/trainer_pt_utils.py", line 337, in _nested_set_tensors
    slice_len = self._nested_set_tensors(x, y)
  File "./sim/lib/python3.7/site-packages/transformers/trainer_pt_utils.py", line 337, in _nested_set_tensors
    slice_len = self._nested_set_tensors(x, y)
  File "./sim/lib/python3.7/site-packages/transformers/trainer_pt_utils.py", line 349, in _nested_set_tensors
    i * slice_len : (i + 1) * slice_len
ValueError: could not broadcast input array from shape (512,8,768) into shape (416,8,768)
```shell

Could you please help me? Thanks a lot !
This is a duplicate of #7584 I think. Some workarounds are mentioned in that issue and a fix is on its way.<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
<|||||>This issue has been automatically marked as stale and been closed because it has not had recent activity. Thank you for your contributions.

If you think this still needs to be addressed please comment on this thread.