Thanks @valhalla and @sgugger for the fix!
I leave here a (hopefully) permalink to a notebook to reproduce the issue that I had with 3.3.0 https://colab.research.google.com/gist/davidefiocco/3bbe492033b5675ab03405019a71f9ce/colafinetuning.ipynb 44