Hi @mayanksatnalika
Yes, you can load a pre-trained base model for SequenceClassification.
RobertaForSequenceClassification  adds the classification head itself, you won’t need to do that manually. So you can fine-tune it for classification.