metadata

language: en
datasets:
  - squad_v2
license: cc-by-4.0
tags:
  - deberta
  - deberta-v3

deberta-v3-base for QA

This is the deberta-v3-base model, fine-tuned using the SQuAD2.0 dataset. It's been trained on question-answer pairs, including unanswerable questions, for the task of Question Answering.

Overview

Language model: deberta-v3-base
Language: English
Downstream-task: Extractive QA
Training data: SQuAD 2.0
Eval data: SQuAD 2.0
Code: See an example QA pipeline on Haystack
Infrastructure:

Hyperparameters

batch_size = 12
n_epochs = 4
base_LM_model = "deberta-v3-base"
max_seq_len = 512
learning_rate = 2e-5
lr_schedule = LinearWarmup
warmup_proportion = 0.2
doc_stride = 128
max_query_length = 64

{'EM': 81.90853196327804,
 'f1': 86.86782434367508,
 'top_n_accuracy': 97.75120020213932,
 'top_n': 4,
 'EM_text_answer': 75.94466936572199,
 'f1_text_answer': 85.87747611883509,
 'top_n_accuracy_text_answer': 95.49595141700405,
 'top_n_EM_text_answer': 79.21727395411607,
 'top_n_f1_text_answer': 90.0904505126207,
 'Total_text_answer': 5928,
 'EM_no_answer': 87.85534062237174,
 'f1_no_answer': 87.85534062237174,
 'top_n_accuracy_no_answer': 100.0,
 'Total_no_answer': 5945}