Problem with tokenization using the 'distil-bert-uncased' tokenizer
### System Info

```
- `transformers` version: 4.24.0
- Platform: Windows-10-10.0.19044-SP0
- Python version: 3.10.8
- Huggingface_hub version: 0.11.0
- PyTorch version (GPU?): 1.13.0+cu117 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>
```

### Who can help?

@ArthurZucker, @younesbelkada & @sgugger - since I am facing a tokenization issue with the trainer on `distil-bert-uncased` mostly using the official scripts.

### Information

- [X] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

I was following official script titled [Fine-tune a pretrained model](https://huggingface.co/docs/transformers/training) with the only difference being that I am loading my dataset from a local `.csv` file with structure its described below.

| Text | Label (integers 0-2) |
| - | - |
| string 1 | label 1 |
| string 2 | label 2 |
| string n | label n |

My real dataset however, does not have a header row, It's just strings & integral labels ranging from 0-2. Header row was added for better readability of my structure using markdown.

I am using the library-defined `load_dataset` function to load my csv file right into a Hugging Face dataset. My code is as follows:

```python
from datasets import dataset_dict, load_dataset
from transformers import DistilBertTokenizerFast, DistilBertModel, Trainer, TrainingArguments
import torch

DATA_PATH = "SOMEPATH"

dataset = load_dataset('csv', data_files={'train': f"{DATA_PATH}\\train.csv", 'test': f"{DATA_PATH}\\test.csv", 'validation': f"{DATA_PATH}\\validation.csv"}, column_names=['text', 'label'], split=['train', 'test', 'validation'])
dataset = dataset_dict.DatasetDict({'train':dataset[0], 'test':dataset[1], 'validation':dataset[2]})

tokenizer = DistilBertTokenizerFast.from_pretrained('distilbert-base-uncased')

def tokenize_function(examples):
    return tokenizer(examples["text"], padding=True, truncation=True, max_length=512)

FINAL_DS = dataset.map(tokenize_function, batched=True)

training_stuff = {
    "batch_size": 64, 
    "epochs": 4, 
    "learning_rate": 1e-5,
    "weight_decay": 0.01
    }

training_args = TrainingArguments(
    output_dir="/Models/DistilBert",
    per_device_train_batch_size=training_stuff["batch_size"],
    evaluation_strategy="steps",
    num_train_epochs=training_stuff["epochs"],
    fp16=True,
    save_steps=100,
    eval_steps=50,
    logging_steps=10,
    weight_decay=training_stuff["weight_decay"],
    learning_rate=training_stuff["learning_rate"],
    save_total_limit=64,
    remove_unused_columns=False,
    push_to_hub=False,
    report_to='tensorboard',
    load_best_model_at_end=True,
)

model = DistilBertModel.from_pretrained(
    'distilbert-base-uncased',
    num_labels=3,
    id2label={0: 'Biased', 1: 'Non-biased', 2: 'No agreemnt'},
    label2id={'Biased': 0, 'Non-biased': 1, 'No agreement': 2},
    )

trainer = Trainer(
            model=model,
            args=training_args,
            train_dataset=FINAL_DS['train'],
            eval_dataset=FINAL_DS['validation'],
            tokenizer=tokenizer,
        )

train_results = trainer.train()
trainer.save_model()

```


However, when I run this script (at `trainer.train()`), I get the following error.
<img width="745" alt="err1" src="https://user-images.githubusercontent.com/67118602/216493803-c8d640f0-d3e8-4116-9f75-c4379ce8a290.png">

![err3](https://user-images.githubusercontent.com/67118602/216495983-64afbdd9-0a5e-40ef-b91d-dcf184b88913.png)

The hugging face forum [link](https://discuss.huggingface.co/t/getting-a-value-error-unable-to-create-a-tensor-because-the-feature-text-has-excessive-nesting-and-it-expects-it-to-be-int-for-some-reason/30890?u=quantumstatic) to this issue, unfortunately with no responses.






### Expected behavior

I would expect the model to start training.
You are not dropping the text columns of your dataset, so the Trainer is then unable to make tensors out of them. You need to either remove the problematic columns of the dataset or remove the `remove_unused_columns=False` argument.<|||||>Could you please explain how removing the text column will help? Wouldn't the transformer need text column for text2text generation. Even if am failing to grasp the idea of dropping the column and `remove_unused_columns=True` is the correct way to move forward. I get the following error:

![err4](https://user-images.githubusercontent.com/67118602/216628779-d4e9260f-862b-4d02-8d5d-4529b64b12be.png)

Why would the model not generate outputs? I double checked that my `input_ids` & `attention_mask` they are sufficiently sized according to the input data.<|||||>You should really go on the [forums](https://discuss.huggingface.co/) to help debug your code as the wider community will be there to help. Your code fine-tunes the base model, there is no text2text class here and no `labels` to go with it. That's why you get this error message, there is no loss to optimize.<|||||>I am sorry, I don't want hugging face to debug my code. I completely understand your time is more valuable than debugging my code. However, I am a beginner and I did try to look up online and I did try the forums and I assure you this is my last option to get help. 

In addition, I thank you once again for helping me with such a trivial problem. <|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.