What is the f1 score of Squad v2.0 on bert-base? I only got f1 score 74.78.
## ❓ Questions & Help

<!-- A clear and concise description of the question. -->
Hello, I am doing some experiment of squad v2.0 on bert-base (NOT bert-large).
According to the BERT paper, bert-large achieves f1 score 81.9 with squad v2.0.
Since I couldn't find the official result for bert-base, I am not sure if I am getting the right f1 score.
Has anyone tried running squad v2.0 on bert base? 

I got f1 score **74.78** for squad v2.0 result on bert-base, using below command:
sudo python3 ../../../run_squad.py \
  --model_type bert \
  --model_name_or_path bert-base-cased \
  --do_train \
  --do_eval \
  --train_file $SQUAD2_DIR/train-v2.0.json \
  --predict_file $SQUAD2_DIR/dev-v2.0.json \
  --per_gpu_train_batch_size 4 \
  --learning_rate 4e-5 \
  --num_train_epochs 2.0 \
  --max_seq_length 384 \
  --doc_stride 128 \
  --version_2_with_negative \
  --overwrite_output_dir \
  --output_dir ../../../bert_base/$TASK_NAME/
Your result is fine. This [poster](https://web.stanford.edu/class/cs224n/posters/15848021.pdf) says that they achieved **76.70**. Maybe you can get there as well when you train for 2 more epochs.<|||||>Thank you for your reply! :)<|||||>Please close the question if the answer suits your needs.<|||||>Sorry, I closed it!