I am not sure, this is my first time dealing with a bert trained model. But here is my colab notebook, if you’d like to take a look https://colab.research.google.com/drive/1hfkGaZvSLkxB4BqUm24E5RiF3V098u4m?usp=sharing 2.