ValueError: Expected input batch_size (8) to match target batch_size (1008).
### System Info

- `transformers` version: 4.24.0
- Platform: Linux-5.10.133+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.7.15
- Huggingface_hub version: 0.11.1
- PyTorch version (GPU?): 1.12.1+cu113 (True)
- Tensorflow version (GPU?): 2.9.2 (True)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

### Who can help?

@sgugger @lys

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

I am fine-tuning a custom model for multiclass classification. When I run the sixth cell of [this collab notebook](https://colab.research.google.com/drive/1NYk_RJcZ3GmwYQTv9X3kcG_FbHDjWvsd?usp=sharing), I get the following error: 

```
ValueError                                Traceback (most recent call last)

[<ipython-input-17-f9d56f5f4088>](https://localhost:8080/#) in <module>
     27   )
     28 
---> 29 trainer.train()
     30 
     31 trainer.push_to_hub()

8 frames

[/usr/local/lib/python3.7/dist-packages/transformers/trainer.py](https://localhost:8080/#) in train(self, resume_from_checkpoint, trial, ignore_keys_for_eval, **kwargs)
   1503             resume_from_checkpoint=resume_from_checkpoint,
   1504             trial=trial,
-> 1505             ignore_keys_for_eval=ignore_keys_for_eval,
   1506         )
   1507 

[/usr/local/lib/python3.7/dist-packages/transformers/trainer.py](https://localhost:8080/#) in _inner_training_loop(self, batch_size, args, resume_from_checkpoint, trial, ignore_keys_for_eval)
   1747                         tr_loss_step = self.training_step(model, inputs)
   1748                 else:
-> 1749                     tr_loss_step = self.training_step(model, inputs)
   1750 
   1751                 if (

[/usr/local/lib/python3.7/dist-packages/transformers/trainer.py](https://localhost:8080/#) in training_step(self, model, inputs)
   2506 
   2507         with self.compute_loss_context_manager():
-> 2508             loss = self.compute_loss(model, inputs)
   2509 
   2510         if self.args.n_gpu > 1:

[/usr/local/lib/python3.7/dist-packages/transformers/trainer.py](https://localhost:8080/#) in compute_loss(self, model, inputs, return_outputs)
   2538         else:
   2539             labels = None
-> 2540         outputs = model(**inputs)
   2541         # Save past state if it exists
   2542         # TODO: this needs to be fixed and made cleaner later.

[/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py](https://localhost:8080/#) in _call_impl(self, *input, **kwargs)
   1128         if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
   1129                 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1130             return forward_call(*input, **kwargs)
   1131         # Do not call functions when jit is used
   1132         full_backward_hooks, non_full_backward_hooks = [], []

[/usr/local/lib/python3.7/dist-packages/transformers/models/roberta/modeling_roberta.py](https://localhost:8080/#) in forward(self, input_ids, attention_mask, token_type_ids, position_ids, head_mask, inputs_embeds, labels, output_attentions, output_hidden_states, return_dict)
   1238             elif self.config.problem_type == "single_label_classification":
   1239                 loss_fct = CrossEntropyLoss()
-> 1240                 loss = loss_fct(logits.view(-1, self.num_labels), labels.view(-1))
   1241             elif self.config.problem_type == "multi_label_classification":
   1242                 loss_fct = BCEWithLogitsLoss()

[/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py](https://localhost:8080/#) in _call_impl(self, *input, **kwargs)
   1128         if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
   1129                 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1130             return forward_call(*input, **kwargs)
   1131         # Do not call functions when jit is used
   1132         full_backward_hooks, non_full_backward_hooks = [], []

[/usr/local/lib/python3.7/dist-packages/torch/nn/modules/loss.py](https://localhost:8080/#) in forward(self, input, target)
   1164         return F.cross_entropy(input, target, weight=self.weight,
   1165                                ignore_index=self.ignore_index, reduction=self.reduction,
-> 1166                                label_smoothing=self.label_smoothing)
   1167 
   1168 

[/usr/local/lib/python3.7/dist-packages/torch/nn/functional.py](https://localhost:8080/#) in cross_entropy(input, target, weight, size_average, ignore_index, reduce, reduction, label_smoothing)
   3012     if size_average is not None or reduce is not None:
   3013         reduction = _Reduction.legacy_get_string(size_average, reduce)
-> 3014     return torch._C._nn.cross_entropy_loss(input, target, weight, _Reduction.get_enum(reduction), ignore_index, label_smoothing)
   3015 
   3016 

ValueError: Expected input batch_size (8) to match target batch_size (1008).
```



### Expected behavior

I expected training to continue as usual.
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.<|||||>Disabling one-hot encoding (`encoding['labels'] = [[stage] for stage in examples['Stage']]`) and the data collator (`# data_collator=data_collator`) seems to resolve this issue.

Not sure if labels should be sparsely encoded and what the data_collator does to create this error, maybe it 'collates' on the wrong axis?