transformers pytorch tensorboard safetensors bert question-answering generated_from_trainer dataset:squad license:apache-2.0 endpoints_compatible region:us
---
datasets:
- squad
license: apache-2.0
tags:
- generated_from_trainer
model-index:
- name: bert-finetuned-squad
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# bert-finetuned-squad

This model is a fine-tuned version of [bert-base-cased](https://huggingface.co/bert-base-cased) on the squad dataset.

## Model description

The BERT fine-tuned SQuAD model is a version of the BERT (Bidirectional Encoder Representations from Transformers) model that has been fine-tuned on the Stanford Question Answering Dataset (SQuAD). It is designed to answer questions based on the context given. The SQuAD dataset is a collection of 100k+ questions and answers based on Wikipedia articles. Fine-tuning the model on this dataset allows it to provide precise answers to a wide array of questions based on a given context.

## Intended uses & limitations

This model is intended to be used for question-answering tasks. Given a question and a context (a piece of text containing the information to answer the question), the model will return the text span in the context that most likely contains the answer. This model is not intended to generate creative content, conduct sentiment analysis, or predict future events.

It's important to note that the model's accuracy is heavily dependent on the relevance and quality of the context it is provided. If the context does not contain the answer to the question, the model will still return a text span, which may not make sense. Additionally, the model may struggle with nuanced or ambiguous questions as it may not fully understand the subtleties of human language.

## Training and evaluation data

The model was trained on the SQuAD dataset, encompassing over 87,599 questions generated by crowd workers from various Wikipedia articles. The answers are text segments from the relevant reading passage. For evaluation, a distinct subset of the SQuAD, containing 10,570 instances, unseen by the model during training, was employed.

## Training procedure

The model was initially pretrained on a large corpus of text in an unsupervised manner, learning to predict masked tokens in a sentence. The pretraining was done on the bert-base-cased model, which was trained on English text in a case-sensitive manner. After this, the model was fine-tuned on the SQuAD dataset. During fine-tuning, the model was trained to predict the start and end positions of the answer in the context text given a question.

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3

### Training results

- exact_match: 81.0406811731315
- f1: 88.65884513439593

### Framework versions

- Transformers 4.30.2
- Pytorch 2.0.1+cu118
- Datasets 2.13.1
- Tokenizers 0.13.3
