Hi, there hasn’t been any answer to my report, maybe, I have posted it to the wrong place or haven’t tagged the correct person? Could someone please help me get the message through?
Also, in case this helps, here are some steps, that someone can take to reproduce the incorrect model behaviour:

Download the default ‘flaubert-base-uncased’ model from the official repo, fix the seed, start the training and print out something (train loss, some weights, ets).
Then in the same downloaded model, modify the "do_lower_case": false and restart the training. Make sure that the printed value is exactly the same.
Then still in the same downloaded model, modify the option to "do_lowercase": false, restart the training and once again make sure that the printed value is the same.

Basically, those three trainings are the same because in the first two cases the network uses the default value of "do_lowercase": false, while in the third one we explicitly select the option.

Finally, in the very same model, set "do_lowercase": true, launch the training and check that now the training is different, and that this is indeed the correct option name, which controls the upper/lower case of the model.