readme: datasets:
  - squad
  - newsqa
  - hotpot_qa
  - biu-nlp/qamr
  - search_qa
  - natural_questions
  - trivia_qa
  - duorc
  language:
  - en
  metrics:
  - squad
  ---
  
  # Model Card for Model ID
  
  <!-- Provide a quick summary of what the model is/does. -->
  
  Checkpoint of MetaQA trained only on extractive QA datasets from MetaQA: Combining Expert Agents for Multi-Skill Question Answering (https://arxiv.org/abs/2112.01922)
  
  ## Evaluation Results
  ```
  {
      "SQuAD": {
          "exact_match": 86.73139158576052,
          "f1": 92.65156746563402
      },
      "NewsQA": {
          "exact_match": 55.84045584045584,
          "f1": 71.73547617592037
      },
      "HotpotQA": {
          "exact_match": 64.8135593220339,
          "f1": 79.61023604916922
      },
      "SearchQA": {
          "exact_match": 75.04122497055359,
          "f1": 81.37280639135817
      },
      "NaturalQuestionsShort": {
          "exact_match": 69.50763477718915,
          "f1": 81.30374741690376
      },
      "TriviaQA-web": {
          "exact_match": 77.18396711202466,
          "f1": 81.52989853015538
      },
      "QAMR": {
          "exact_match": 72.07531203723292,
          "f1": 83.9068616637681
      },
      "DuoRC": {
          "exact_match": 39.35626573106552,
          "f1": 51.033295034422466
      }
  }
  ```