Thank you @lewtun.
So are you saying that tokenized adds [CLS]?
Also, I am freezing the BERT layer but fine-tuning the classification layer. The reason I am doing that is because Colab crashes as it runs out of GPU if I don’t. If I try using just the CPU, I get “Your session crashed after using all available RAM.”
I therefore add
for param in model.bert.parameters():
        param.requires_grad = False

Which fixes that, but then the model doesn’t seem to learn anything