readme: ---
  license: apache-2.0
  tags:
  - generated_from_trainer
  - course
  - fine-tuned
  - bert
  datasets:
  - squad
  model-index:
  - name: bert-finetuned-squad
    results: []
  ---
  
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
  should probably proofread and complete it, then remove this comment. -->
  
  # bert-finetuned-squad
  
  This model is a fine-tuned version of [bert-base-cased](https://huggingface.co/bert-base-cased) on the squad dataset. 
  
  ## Model description
  
  The model is based on a pretrained model where fine-tuning was done to setup the model for the actual task (use case). 
  
  ## Intended uses & limitations
  
  Intended use here is to create a working model to teach myself and others about how to use transformers for Questions/Answering tasks.
  The limitation is that this approach expects input data to always have __context, question, answer__ and to use the syntax [MASK] for this use case.
  
  ## Training and evaluation data
  Training took more time than expected given use case. 
  
  ## Training procedure
  
  Setup files with the expected format from pretrained model (__context, question, answer__)
  `display("Context: ", raw_datasets["train"][0]["context"])`
  `display("Question: ", raw_datasets["train"][0]["question"])`
  `display("Answer: ", raw_datasets["train"][0]["answers"])`
  
  > output for code above
  > 
  ```json
  __Context  :__ Architecturally, the school has a Catholic character. Atop the Main Building\'s gold dome is a golden statue of the Virgin Mary. Immediately in front of the Main Building and facing it, is a copper statue of Christ with arms upraised with the legend "Venite Ad Me Omnes". Next to the Main Building is the Basilica of the Sacred Heart. Immediately behind the basilica is the Grotto, a Marian place of prayer and reflection. It is a replica of the grotto at Lourdes, France where the Virgin Mary reputedly appeared to Saint Bernadette Soubirous in 1858. At the end of the main drive (and in a direct line that connects through 3 statues and the Gold Dome), is a simple, modern stone statue of Mary.
  __Question :__ To whom did the Virgin Mary allegedly appear in 1858 in Lourdes France?
  __Answer   :__ {'text': ['Saint Bernadette Soubirous'], 'answer_start': [515]}
  ```
  
  ### Training hyperparameters
  
  The following hyperparameters were used during training:
  - learning_rate: 2e-05
  - train_batch_size: 8
  - eval_batch_size: 8
  - seed: 42
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  - lr_scheduler_type: linear
  - num_epochs: 3
  - mixed_precision_training: Native AMP
  
  ### Training results
  
  
  
  ### Framework versions
  
  - Transformers 4.25.1
  - Pytorch 1.13.1+cu116
  - Datasets 2.8.0
  - Tokenizers 0.13.2