readme: ---
  license: apache-2.0
  tags:
  - generated_from_trainer
  metrics:
  - f1
  model-index:
  - name: 8_koelectra_train_korquad-1_2_aihub
    results: []
  ---
  
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
  should probably proofread and complete it, then remove this comment. -->
  
  # 8_koelectra_train_korquad-1_2_aihub
  
  This model is a fine-tuned version of [monologg/koelectra-base-v3-discriminator](https://huggingface.co/monologg/koelectra-base-v3-discriminator) on the None dataset.
  It achieves the following results on the evaluation set:
  - Exact Match: 78.9613
  - F1: 84.5790
  - Loss: 0.8506
  
  ## Model description
  
  More information needed
  
  ## Intended uses & limitations
  
  More information needed
  
  ## Training and evaluation data
  
  More information needed
  
  ## Training procedure
  
  ### Training hyperparameters
  
  The following hyperparameters were used during training:
  - learning_rate: 1e-05
  - train_batch_size: 8
  - eval_batch_size: 8
  - seed: 42
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  - lr_scheduler_type: linear
  - lr_scheduler_warmup_ratio: 0.1
  - num_epochs: 10
  - mixed_precision_training: Native AMP
  - label_smoothing_factor: 0.1
  
  ### Training results
  
  | Training Loss | Epoch | Step   | Exact Match | F1      | Validation Loss |
  |:-------------:|:-----:|:------:|:-----------:|:-------:|:---------------:|
  | 1.5282        | 0.33  | 4000   | 64.4807     | 72.3547 | 1.4717          |
  | 1.0638        | 0.67  | 8000   | 72.1118     | 79.0236 | 1.0938          |
  | 1.0703        | 1.0   | 12000  | 74.0859     | 80.5242 | 1.0459          |
  | 0.9242        | 1.34  | 16000  | 75.1325     | 81.4470 | 0.9775          |
  | 0.9312        | 1.67  | 20000  | 75.6492     | 81.7357 | 0.9707          |
  | 0.9483        | 2.01  | 24000  | 76.2189     | 82.3461 | 0.9248          |
  | 0.8454        | 2.34  | 28000  | 76.8813     | 82.9913 | 0.9268          |
  | 0.8541        | 2.67  | 32000  | 77.1330     | 83.1591 | 0.9004          |
  | 0.8647        | 3.01  | 36000  | 77.1860     | 83.1519 | 0.8911          |
  | 0.8952        | 3.34  | 40000  | 77.1993     | 83.1777 | 0.8765          |
  | 0.7345        | 3.68  | 44000  | 77.3450     | 83.4184 | 0.9365          |
  | 0.708         | 4.01  | 48000  | 77.8617     | 83.7737 | 0.8599          |
  | 0.7217        | 4.34  | 52000  | 77.8352     | 83.6681 | 0.8770          |
  | 0.817         | 4.68  | 56000  | 77.9809     | 83.8054 | 0.8730          |
  | 0.7655        | 5.01  | 60000  | 78.0207     | 83.8704 | 0.8623          |
  | 0.7276        | 5.35  | 64000  | 78.2989     | 84.0245 | 0.8535          |
  | 0.6739        | 5.68  | 68000  | 78.2724     | 84.0880 | 0.8726          |
  | 0.652         | 6.02  | 72000  | 78.5639     | 84.2059 | 0.8657          |
  | 0.6615        | 6.35  | 76000  | 78.3254     | 84.1279 | 0.8623          |
  | 0.6624        | 6.68  | 80000  | 78.7493     | 84.4215 | 0.8525          |
  | 0.707         | 7.02  | 84000  | 78.5374     | 84.2300 | 0.8486          |
  | 0.8086        | 7.35  | 88000  | 78.3519     | 84.1909 | 0.8442          |
  | 0.6347        | 7.69  | 92000  | 78.6963     | 84.4347 | 0.8760          |
  | 0.702         | 8.02  | 96000  | 78.9083     | 84.6330 | 0.8418          |
  | 0.6618        | 8.36  | 100000 | 78.7493     | 84.5021 | 0.8672          |
  | 0.6294        | 8.69  | 104000 | 78.5374     | 84.3771 | 0.8770          |
  | 0.5797        | 9.02  | 108000 | 78.5904     | 84.3051 | 0.8623          |
  | 0.6073        | 9.36  | 112000 | 78.9216     | 84.6703 | 0.8638          |
  | 0.6717        | 9.69  | 116000 | 78.9613     | 84.5790 | 0.8506          |
  
  
  ### Framework versions
  
  - Transformers 4.25.1
  - Pytorch 1.7.1
  - Datasets 2.7.0
  - Tokenizers 0.13.2
  