Can you verify that you prepared the data correctly for the model?
What I typically do is check random elements of the dataset, i.e.
encoding = train_dataset[0]

then verify some things like:
for k,v in encoding.items():
     print(k, v.shape)

I also decode the input_ids of an example back to text to see whether it’s created correctly:
from transformers import BertTokenizer

tokenizer = BertTokenizer.from_pretrained("bert-base-uncased")
tokenizer.decode(encoding["input_ids"])

And check the corresponding label. Also, to check whether batches are created correctly, I typically create a dataloader, and verify some more:
from torch.utils.data import DataLoader

train_dataloader  = DataLoader(train_dataset, batch_size=4, shuffle=True)
batch = next(iter(train_dataloader))
for k,v in batch.items():
     print(k, v.shape)

The Trainer automatically batches examples together, using the default_data_collator.