Hi @tejan-rgb! This is the expected behavior as bert-base-cased is trained without a task-specific head, and its goal is to learn powerful general representations of language. The resulting message you are seeing is expected, as it is suggested that you “fine-tune” this task-specific head (classification in your case) to your specific data and labels. If you have data you’d like to specialize this model to, then you can ignore the warning messages and train the model. If you want an out of the box model for a sequence classification task, thankfully Hugging Face’s awesome community has already fine-tuned a ton of models - you’ll need to download the weights of the model that best suits your needs here 11.
Need more comprehensive support specialized to your use-cases?  has you covered! Through our Expert Acceleration Program 1 your business can leverage our expertise to accelerate your NLP roadmap, from modeling to production.