I am not familiar with BERT, but here are a couple of things you could try to fix this:

See if you are missing some normalization steps in input processing. Specifically, a normalization step that makes all the input numbers small. An example of such a step from computer vision would be: dividing all pixel values by 255 so that they are all at most 1.


I really appreciate what you are doing with me. Thanks!!
indeed I did not do any normalization nor preprocessing to my data I just take them as they are for tweeter with their emojis and all the characters that you can find in comments! but I don’t think that is the problem no? Camembert is trained with 148GB of text! well, I am not sure but I just supposed that!


More generally: many networks work best when we scale the input features so that they are all within a comparable range. Is there such a step involved in training BERT? If so, have you done this step?


Yes there is I already put it in the parameters of the model
encoded_data_train = tokenizer.batch_encode_plus(msgTrain,
                                                 add_special_tokens=True, 
                                                 return_attention_mask = True,
                                                 max_length=50, #Here it is it can be mac 512 and I already change it 
                                                   #many time but it gets the model training very slowly
                                                 padding=True, 
                                                 truncation=True, 
                                                 return_tensors = 'pt'
                                                 )



Reduce the initial learning rate till your net starts finding more than one class. From your code above you have lr set to 1e-3. Try reducing it to 1e-4, then 1e-5, and so on, till more classes start appearing in the predictions. Finding the right learning rate may require some trial and error.


I also did this   but I got the same results!

Edited to add: From the denominator it looks like all your validation instances have their (correct/ground truth) label set to “Neutral”. This does not look like a good thing for multiclass validation. Usually we want the validation set to have all classes in the same proportion (as far as possible) as the training set.

I am not sure what do you mean. can you please explain more ?
Thank you very much!!