ValueError: Unable to create tensor, you should probably activate truncation and/or padding with 'padding=True' 'truncation=True' to have batched tensors with the same length. Perhaps your features (`labels` in this case) have excessive nesting (inputs type `list` where type `int` is expected).
### System Info

- `transformers` version: 4.25.1
- Platform: Linux-5.10.133+-x86_64-with-glibc2.27
- Python version: 3.8.15
- Huggingface_hub version: 0.11.1
- PyTorch version (GPU?): 1.12.1+cu113 (True)
- Tensorflow version (GPU?): 2.9.2 (True)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: yes (Tesla T4)
- Using distributed or parallel set-up in script?: no

### Who can help?

@sgugger maybe you could help?

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction


# Information
I am using the implementation of text classification given in official [documentation ](https://huggingface.co/docs/transformers/tasks/sequence_classification)from huggingface and one given by @lewtun in his book. 
I retrained an instance of sentence-transformers using contrastive loss on an unsupervised data dump and now want to finetune the above model on a labeled, binary dataset. 
[This ](https://github.com/huggingface/transformers/issues/15505)issue is similar, and I followed the fix but to no help.

# To reproduce

1. Run [this notebook](https://colab.research.google.com/drive/1VMl5l1O4lrgSMiGTh4yKIWEY2XGUgSIm?usp=sharing)
2. Trainer.train() should produce the following error:

```
ValueError                                Traceback (most recent call last)
[/usr/local/lib/python3.8/dist-packages/transformers/tokenization_utils_base.py](https://localhost:8080/#) in convert_to_tensors(self, tensor_type, prepend_batch_axis)
    716                 if not is_tensor(value):
--> 717                     tensor = as_tensor(value)
    718 

ValueError: too many dimensions 'str'

During handling of the above exception, another exception occurred:

ValueError                                Traceback (most recent call last)
9 frames
[<ipython-input-75-ce45916ac715>](https://localhost:8080/#) in <module>
      7 )
      8 
----> 9 trainer.train()

[/usr/local/lib/python3.8/dist-packages/transformers/trainer.py](https://localhost:8080/#) in train(self, resume_from_checkpoint, trial, ignore_keys_for_eval, **kwargs)
   1525             self._inner_training_loop, self._train_batch_size, args.auto_find_batch_size
   1526         )
-> 1527         return inner_training_loop(
   1528             args=args,
   1529             resume_from_checkpoint=resume_from_checkpoint,

[/usr/local/lib/python3.8/dist-packages/transformers/trainer.py](https://localhost:8080/#) in _inner_training_loop(self, batch_size, args, resume_from_checkpoint, trial, ignore_keys_for_eval)
   1747 
   1748             step = -1
-> 1749             for step, inputs in enumerate(epoch_iterator):
   1750 
   1751                 # Skip past any already trained steps if resuming training

[/usr/local/lib/python3.8/dist-packages/torch/utils/data/dataloader.py](https://localhost:8080/#) in __next__(self)
    679                 # TODO(https://github.com/pytorch/pytorch/issues/76750)
    680                 self._reset()  # type: ignore[call-arg]
--> 681             data = self._next_data()
    682             self._num_yielded += 1
    683             if self._dataset_kind == _DatasetKind.Iterable and \

[/usr/local/lib/python3.8/dist-packages/torch/utils/data/dataloader.py](https://localhost:8080/#) in _next_data(self)
    719     def _next_data(self):
    720         index = self._next_index()  # may raise StopIteration
--> 721         data = self._dataset_fetcher.fetch(index)  # may raise StopIteration
    722         if self._pin_memory:
    723             data = _utils.pin_memory.pin_memory(data, self._pin_memory_device)

[/usr/local/lib/python3.8/dist-packages/torch/utils/data/_utils/fetch.py](https://localhost:8080/#) in fetch(self, possibly_batched_index)
     50         else:
     51             data = self.dataset[possibly_batched_index]
---> 52         return self.collate_fn(data)

[/usr/local/lib/python3.8/dist-packages/transformers/data/data_collator.py](https://localhost:8080/#) in __call__(self, features)
    247 
    248     def __call__(self, features: List[Dict[str, Any]]) -> Dict[str, Any]:
--> 249         batch = self.tokenizer.pad(
    250             features,
    251             padding=self.padding,

[/usr/local/lib/python3.8/dist-packages/transformers/tokenization_utils_base.py](https://localhost:8080/#) in pad(self, encoded_inputs, padding, max_length, pad_to_multiple_of, return_attention_mask, return_tensors, verbose)
   3015                 batch_outputs[key].append(value)
   3016 
-> 3017         return BatchEncoding(batch_outputs, tensor_type=return_tensors)
   3018 
   3019     def create_token_type_ids_from_sequences(

[/usr/local/lib/python3.8/dist-packages/transformers/tokenization_utils_base.py](https://localhost:8080/#) in __init__(self, data, encoding, tensor_type, prepend_batch_axis, n_sequences)
    208         self._n_sequences = n_sequences
    209 
--> 210         self.convert_to_tensors(tensor_type=tensor_type, prepend_batch_axis=prepend_batch_axis)
    211 
    212     @property

[/usr/local/lib/python3.8/dist-packages/transformers/tokenization_utils_base.py](https://localhost:8080/#) in convert_to_tensors(self, tensor_type, prepend_batch_axis)
    731                         "Please see if a fast version of this tokenizer is available to have this feature available."
    732                     )
--> 733                 raise ValueError(
    734                     "Unable to create tensor, you should probably activate truncation and/or padding with"
    735                     " 'padding=True' 'truncation=True' to have batched tensors with the same length. Perhaps your"

ValueError: Unable to create tensor, you should probably activate truncation and/or padding with 'padding=True' 'truncation=True' to have batched tensors with the same length. Perhaps your features (`labels` in this case) have excessive nesting (inputs type `list` where type `int` is expected).

```


### Expected behavior

The model should train without failure, 
Please use the [forums](https://discuss.huggingface.co/) to help debug your code. We also have a [step by step guide](https://huggingface.co/course/chapter8/4?fw=pt) to help debug issues with the `Trainer`.

In this instance you did not convert your labels from strings to integers, so the data collator cannot build a batch. Also you shouldn't share your huggingface token in a notebook like this, I recommend you invalidate it :-)<|||||>> Please use the [forums](https://discuss.huggingface.co/) to help debug your code. We also have a [step by step guide](https://huggingface.co/course/chapter8/4?fw=pt) to help debug issues with the `Trainer`.
> 
> In this instance you did not convert your labels from strings to integers, so the data collator cannot build a batch. Also you shouldn't share your huggingface token in a notebook like this, I recommend you invalidate it :-)

Thank you for the suggestions @sgugger 
I do have a quick question - shouldn't the below snippet take care of converting labels to ids and back?
```
    id2label=id2label, 
    label2id=label2id, 
```<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.<|||||>I am having the same issue without labels. It works for the first 24 iterations but then suddenly it stops padding. I have two images `[image1 PIL, image2 PIL]` and two sentences `[sentence1, sentence2]`.
`inputs = preprocess([image1 PIL, image2 PIL], [sentence1, sentence2], return_tensors="pt", padding=True, truncation=True).to(device)`

The first iterations produce the correct output:
`[[101, 1037, 6302, 1997, 1037, 3287, 5093, 1012, 102, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [101, 1037, 6302, 1997, 1037, 2931, 5093, 1012, 102, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]]
`

Then suddenly it does not:
`[[101, 1037, 6302, 1997, 1037, 13755, 2492, 1012, 102], [101, 1037, 6302, 1997, 1037, 3103, 15909, 2492, 1012, 102]]
`

Both have padding='True'. The error is:

`Traceback (most recent call last):
  File "anaconda3/envs/SRI/lib/python3.8/site-packages/transformers/tokenization_utils_base.py", line 718, in convert_to_tensors
    tensor = as_tensor(value)
ValueError: expected sequence of length 9 at dim 1 (got 10)`

I cannot quite figure out why it suddently stops padding to the same lenghth. I have even tried setting the max length and the same thing happens. I have tested the text to make sure there are no changes there and the images. 