MABtt:

I don’t think that is the problem no? Camembert is trained with 148GB of text! well, I am not sure but I just supposed that!


As I said, I am not familiar with how BERT models work. The model may expect that you normalize the input in some manner. The model having been trained on large amounts of data does not mean that you don’t have to preprocess the input. For example, every pre-trained model from TorchVision expects that its input is normalized in a very particular way:
All pre-trained models expect input images normalized in the same way, 
i.e. mini-batches of 3-channel RGB images of shape (3 x H x W), where 
H and W are expected to be at least 224. The images have to be loaded
in to a range of [0, 1] and then normalized using 
`mean = [0.485, 0.456, 0.406]` and `std = [0.229, 0.224, 0.225]`.

Many of these vision models are trained on much more than 148GB of images. Camembert may also expect some such input normalization. You have to check its documentation to see if this is the case.



 MABtt:


Edited to add: From the denominator it looks like all your validation instances have their (correct/ground truth) label set to “Neutral”. This does not look like a good thing for multiclass validation. Usually we want the validation set to have all classes in the same proportion (as far as possible) as the training set.

I am not sure what do you mean. can you please explain more ?


This was  a mistake I made in interpreting the second 7501 in:
Class: Neutral
Accuracy: 7501/7501

After looking over the code which prints this, I think this should be fine (provided the logic there is OK; I didn’t look too closely).
Given that Camembert is a well-regarded pre-trained model, my current hypothesis is that the issue is with how you feed it your input. Most likely, you are missing to enforce some requirements on the input. So I would suggest that you go carefully through the documentation for Camembert, and compare your input code with other known successful examples of applying Camembert.