readme: ---
  license: apache-2.0
  tags:
  - generated_from_trainer
  datasets:
  - squad
  model-index:
  - name: run01-bert-l-uwwm-squadv1.1-sl256-ds128-e2-tbs16
    results: []
  ---
  
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
  should probably proofread and complete it, then remove this comment. -->
  
  # run01-bert-l-uwwm-squadv1.1-sl256-ds128-e2-tbs16
  
  This model is a fine-tuned version of [bert-large-uncased-whole-word-masking](https://huggingface.co/bert-large-uncased-whole-word-masking) on the squad dataset. ONNX and OpenVINO IR are enclosed here.
  
  ## Model description
  
  More information needed
  
  ## Intended uses & limitations
  
  More information needed
  
  ## Training and evaluation data
  
  More information needed
  
  ## Training procedure
  ```bash
  NEPOCH=2
  TBS=16
  EBS=64
  SL=256
  DS=128
  
  cmd="
  python run_qa.py \
      --model_name_or_path ${BASEM} \
      --dataset_name squad \
      --do_eval \
      --do_train \
      --evaluation_strategy steps \
      --eval_steps 500 \
      --learning_rate 3e-5 \
      --fp16 \
      --num_train_epochs $NEPOCH \
      --per_device_eval_batch_size $EBS \
      --per_device_train_batch_size $TBS \
      --max_seq_length $SL \
      --doc_stride $DS \
      --save_steps 1000 \
      --logging_steps 1 \
      --overwrite_output_dir \
      --run_name $RUNID \
      --output_dir $OUTDIR
  "
  ```
  ### Training hyperparameters
  
  The following hyperparameters were used during training:
  - learning_rate: 3e-05
  - train_batch_size: 16
  - eval_batch_size: 64
  - seed: 42
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  - lr_scheduler_type: linear
  - num_epochs: 2.0
  - mixed_precision_training: Native AMP
  
  ### Training results
  Best checkpoint was at step 11500 but it was not saved. This is final checkpoint (12K+).
  ```
    eval_exact_match = 86.9347
    eval_f1          = 93.1359
    eval_samples     =   12097
  ```
  
  ### Framework versions
  
  - Transformers 4.18.0
  - Pytorch 1.11.0+cu113
  - Datasets 2.1.0
  - Tokenizers 0.12.1
  
  
  
  