Environment info


transformers version: 4.17.0.dev0
Platform: Linux-5.4.144±x86_64-with-Ubuntu-18.04-bionic
Python version: 3.7.12
PyTorch version (GPU?): 1.10.0+cu111 (True)
Tensorflow version (GPU?): 2.7.0 (True)
Flax version (CPU?/GPU?/TPU?): not installed (NA)
Jax version: not installed
JaxLib version: not installed
Using GPU in script?:
Using distributed or parallel set-up in script?:

###Models:

GPT-2: @patrickvonplaten, @LysandreJik



Information
Model I am using gpt2:
The problem arises when using:

[run_ner ] the official example scripts:

The tasks I am working on is:
Token classification on ncbi_disease dataset

To reproduce
python run_ner.py
–model_name_or_path gpt2
–dataset_name ncbi_disease
–output_dir /tmp/test-ner
–do_train
–do_eval
The f1-score after fine-tuning for 3 epoch is only 0.6114 whereas roberta-base provides f1 score of 0.8358 on the evaluation dataset.