Hi @Dhanachandra , thanks for starting this thread.
I would expect GPT models to perform worse than, for example, a Roberta model for token classification (aka NER) because of how GPT models are trained. From what I understand, GPT models are causal LMs, meaning they are trained via next token classification. Roberta, on the other hand, is a masked LM, meaning they are trained on predicting masked token in a text. The difference is that GPT models don’t have “access” to the tokens on the right, whereas Roberta does. That makes GPT models great for text generation, but probably not as good for token classification.
Is there a particular reason you need to use GPT model for your NER problem, or could you also use other models like Roberta?
Cheers
Heiko