Thank you for the update but I think there is more to do than that because I found out that the process of tokenizing takes more than 2H just to tokenize a dataset of 60K tweets which is a very bad this to do.
train_encodings = tokenizer(train_texts, truncation=True, padding=True, max_length=512)
valid_encodings = tokenizer(valid_texts, truncation=True, padding=True, max_length=512)

Can you tell me if there is another way to tokenize the data with less time, please?
All the methods that I found and worked with the TrainingArguments and the Trainer do a simple tokenizer for a small data and wok well but with my data, it takes more than 2H 

EDIT: It would be VERY helpful if you could tell me more about your proposed solution, I didn’t get what do you mean by

I would recommend to iterate the Dataset and DataLoader for a full epoch and make sure all samples can be loaded properly.