Hi All.
I already trained an NER (token classification) model on a custom training dataset with 19 classes. You can explore it here marefa-nlp/marefa-ner 1
The base model which I used to fine-tune my model was xlm-roberta-large
I have now a new dataset and need to use the last trained model marefa-nlp/marefa-ner as the base model this time.
The problem is that the last model was trained to predict the class out of 19 classes, while the new dataset is designed for just 6 classes.
I tried to load the model and reset the configurations to the xlm-roberta-large configuration like this
from transformers import AutoModelForTokenClassification, AutoConfig

base_model = "xlm-roberta-large"
ft_model = "marefa-nlp/marefa-ner"

config = AutoConfig.from_pretrained(base_model)

ner_model = AutoModelForTokenClassification.from_pretrained(ft_model, num_labels=19)
ner_model.config = config 

# THEN using the ner_model to train with the new dataset

but seems not working, as it still requires that the head size be 19
===
Does anyone know how to solve this?
Thanks