readme: ---
  tags:
  - generated_from_trainer
  model-index:
  - name: distilbert-base-uncased-continued_training-medqa
    results: []
  ---
  
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
  should probably proofread and complete it, then remove this comment. -->
  
  # distilbert-base-uncased-continued_training-medqa
  
  This model is a fine-tuned version of [Shaier/distilbert-base-uncased-continued_training-medqa](https://huggingface.co/Shaier/distilbert-base-uncased-continued_training-medqa) on an unknown dataset.
  It achieves the following results on the evaluation set:
  - Loss: 0.5389
  
  ## Model description
  
  More information needed
  
  ## Intended uses & limitations
  
  More information needed
  
  ## Training and evaluation data
  
  More information needed
  
  ## Training procedure
  
  ### Training hyperparameters
  
  The following hyperparameters were used during training:
  - learning_rate: 2e-05
  - train_batch_size: 64
  - eval_batch_size: 64
  - seed: 42
  - gradient_accumulation_steps: 8
  - total_train_batch_size: 512
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  - lr_scheduler_type: linear
  - lr_scheduler_warmup_steps: 100
  - num_epochs: 220
  - mixed_precision_training: Native AMP
  
  ### Training results
  
  | Training Loss | Epoch | Step  | Validation Loss |
  |:-------------:|:-----:|:-----:|:---------------:|
  | No log        | 1.0   | 333   | 0.4516          |
  | No log        | 2.0   | 666   | 0.4277          |
  | No log        | 3.0   | 999   | 0.3734          |
  | No log        | 4.0   | 1332  | 0.4083          |
  | No log        | 5.0   | 1665  | 0.4134          |
  | No log        | 6.0   | 1998  | 0.5093          |
  | No log        | 7.0   | 2331  | 0.4639          |
  | 0.4564        | 8.0   | 2664  | 0.5132          |
  | 0.4564        | 9.0   | 2997  | 0.3483          |
  | 0.4564        | 10.0  | 3330  | 0.4174          |
  | 0.4564        | 11.0  | 3663  | 0.4975          |
  | 0.4564        | 12.0  | 3996  | 0.4030          |
  | 0.4564        | 13.0  | 4329  | 0.4476          |
  | 0.4564        | 14.0  | 4662  | 0.3692          |
  | 0.4564        | 15.0  | 4995  | 0.4474          |
  | 0.4533        | 16.0  | 5328  | 0.3289          |
  | 0.4533        | 17.0  | 5661  | 0.4647          |
  | 0.4533        | 18.0  | 5994  | 0.4873          |
  | 0.4533        | 19.0  | 6327  | 0.5323          |
  | 0.4533        | 20.0  | 6660  | 0.4273          |
  | 0.4533        | 21.0  | 6993  | 0.3426          |
  | 0.4533        | 22.0  | 7326  | 0.3892          |
  | 0.4533        | 23.0  | 7659  | 0.4297          |
  | 0.4493        | 24.0  | 7992  | 0.4162          |
  | 0.4493        | 25.0  | 8325  | 0.4424          |
  | 0.4493        | 26.0  | 8658  | 0.4575          |
  | 0.4493        | 27.0  | 8991  | 0.4192          |
  | 0.4493        | 28.0  | 9324  | 0.4151          |
  | 0.4493        | 29.0  | 9657  | 0.4321          |
  | 0.4493        | 30.0  | 9990  | 0.4129          |
  | 0.4493        | 31.0  | 10323 | 0.4869          |
  | 0.4456        | 32.0  | 10656 | 0.4510          |
  | 0.4456        | 33.0  | 10989 | 0.5263          |
  | 0.4456        | 34.0  | 11322 | 0.3908          |
  | 0.4456        | 35.0  | 11655 | 0.5016          |
  | 0.4456        | 36.0  | 11988 | 0.4454          |
  | 0.4456        | 37.0  | 12321 | 0.4011          |
  | 0.4456        | 38.0  | 12654 | 0.4714          |
  | 0.4456        | 39.0  | 12987 | 0.4972          |
  | 0.443         | 40.0  | 13320 | 0.4200          |
  | 0.443         | 41.0  | 13653 | 0.4659          |
  | 0.443         | 42.0  | 13986 | 0.4758          |
  | 0.443         | 43.0  | 14319 | 0.4509          |
  | 0.443         | 44.0  | 14652 | 0.4211          |
  | 0.443         | 45.0  | 14985 | 0.4007          |
  | 0.443         | 46.0  | 15318 | 0.3205          |
  | 0.443         | 47.0  | 15651 | 0.4479          |
  | 0.4402        | 48.0  | 15984 | 0.4723          |
  | 0.4402        | 49.0  | 16317 | 0.4956          |
  | 0.4402        | 50.0  | 16650 | 0.4103          |
  | 0.4402        | 51.0  | 16983 | 0.4234          |
  | 0.4402        | 52.0  | 17316 | 0.4052          |
  | 0.4402        | 53.0  | 17649 | 0.4033          |
  | 0.4402        | 54.0  | 17982 | 0.4139          |
  | 0.4402        | 55.0  | 18315 | 0.3618          |
  | 0.4372        | 56.0  | 18648 | 0.5102          |
  | 0.4372        | 57.0  | 18981 | 0.4166          |
  | 0.4372        | 58.0  | 19314 | 0.4475          |
  | 0.4372        | 59.0  | 19647 | 0.4259          |
  | 0.4372        | 60.0  | 19980 | 0.4018          |
  | 0.4372        | 61.0  | 20313 | 0.5005          |
  | 0.4372        | 62.0  | 20646 | 0.4445          |
  | 0.4372        | 63.0  | 20979 | 0.4280          |
  | 0.434         | 64.0  | 21312 | 0.4533          |
  | 0.434         | 65.0  | 21645 | 0.3672          |
  | 0.434         | 66.0  | 21978 | 0.4726          |
  | 0.434         | 67.0  | 22311 | 0.4084          |
  | 0.434         | 68.0  | 22644 | 0.4508          |
  | 0.434         | 69.0  | 22977 | 0.3746          |
  | 0.434         | 70.0  | 23310 | 0.4703          |
  | 0.43