readme: ---
  tags:
  - generated_from_trainer
  model-index:
  - name: DNADebertaK6b
    results: []
  ---
  
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
  should probably proofread and complete it, then remove this comment. -->
  
  # DNADebertaK6b
  
  This model is a fine-tuned version of [](https://huggingface.co/) on the None dataset.
  It achieves the following results on the evaluation set:
  - Loss: 1.4362
  
  ## Model description
  
  More information needed
  
  ## Intended uses & limitations
  
  More information needed
  
  ## Training and evaluation data
  
  More information needed
  
  ## Training procedure
  
  ### Training hyperparameters
  
  The following hyperparameters were used during training:
  - learning_rate: 5e-05
  - train_batch_size: 64
  - eval_batch_size: 64
  - seed: 42
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  - lr_scheduler_type: linear
  - num_epochs: 15
  - mixed_precision_training: Native AMP
  
  ### Training results
  
  | Training Loss | Epoch | Step    | Validation Loss |
  |:-------------:|:-----:|:-------:|:---------------:|
  | 4.061         | 0.25  | 20000   | 1.7733          |
  | 1.7344        | 0.5   | 40000   | 1.6608          |
  | 1.6651        | 0.75  | 60000   | 1.6319          |
  | 1.6359        | 0.99  | 80000   | 1.6092          |
  | 1.6131        | 1.24  | 100000  | 1.5932          |
  | 1.5959        | 1.49  | 120000  | 1.5753          |
  | 1.5827        | 1.74  | 140000  | 1.5624          |
  | 1.5719        | 1.99  | 160000  | 1.5534          |
  | 1.5617        | 2.24  | 180000  | 1.5454          |
  | 1.5551        | 2.49  | 200000  | 1.5403          |
  | 1.5477        | 2.74  | 220000  | 1.5322          |
  | 1.5414        | 2.98  | 240000  | 1.5262          |
  | 1.5366        | 3.23  | 260000  | 1.5220          |
  | 1.5308        | 3.48  | 280000  | 1.5184          |
  | 1.5274        | 3.73  | 300000  | 1.5121          |
  | 1.5224        | 3.98  | 320000  | 1.5085          |
  | 1.5194        | 4.23  | 340000  | 1.5050          |
  | 1.5164        | 4.48  | 360000  | 1.5027          |
  | 1.5126        | 4.72  | 380000  | 1.4984          |
  | 1.5086        | 4.97  | 400000  | 1.4947          |
  | 1.5048        | 5.22  | 420000  | 1.4914          |
  | 1.5025        | 5.47  | 440000  | 1.4914          |
  | 1.5006        | 5.72  | 460000  | 1.4877          |
  | 1.4982        | 5.97  | 480000  | 1.4840          |
  | 1.4952        | 6.22  | 500000  | 1.4825          |
  | 1.4926        | 6.46  | 520000  | 1.4800          |
  | 1.4907        | 6.71  | 540000  | 1.4778          |
  | 1.4886        | 6.96  | 560000  | 1.4761          |
  | 1.4864        | 7.21  | 580000  | 1.4746          |
  | 1.4854        | 7.46  | 600000  | 1.4730          |
  | 1.484         | 7.71  | 620000  | 1.4709          |
  | 1.4826        | 7.96  | 640000  | 1.4676          |
  | 1.4794        | 8.21  | 660000  | 1.4674          |
  | 1.479         | 8.45  | 680000  | 1.4658          |
  | 1.4777        | 8.7   | 700000  | 1.4661          |
  | 1.4751        | 8.95  | 720000  | 1.4649          |
  | 1.4742        | 9.2   | 740000  | 1.4614          |
  | 1.4728        | 9.45  | 760000  | 1.4602          |
  | 1.472         | 9.7   | 780000  | 1.4603          |
  | 1.4703        | 9.95  | 800000  | 1.4577          |
  | 1.4694        | 10.19 | 820000  | 1.4578          |
  | 1.4662        | 10.44 | 840000  | 1.4557          |
  | 1.4668        | 10.69 | 860000  | 1.4545          |
  | 1.466         | 10.94 | 880000  | 1.4548          |
  | 1.465         | 11.19 | 900000  | 1.4513          |
  | 1.4626        | 11.44 | 920000  | 1.4511          |
  | 1.4616        | 11.69 | 940000  | 1.4509          |
  | 1.4609        | 11.93 | 960000  | 1.4485          |
  | 1.4595        | 12.18 | 980000  | 1.4474          |
  | 1.4588        | 12.43 | 1000000 | 1.4470          |
  | 1.4588        | 12.68 | 1020000 | 1.4452          |
  | 1.4565        | 12.93 | 1040000 | 1.4443          |
  | 1.4556        | 13.18 | 1060000 | 1.4433          |
  | 1.4543        | 13.43 | 1080000 | 1.4409          |
  | 1.453         | 13.68 | 1100000 | 1.4409          |
  | 1.4524        | 13.92 | 1120000 | 1.4397          |
  | 1.4511        | 14.17 | 1140000 | 1.4402          |
  | 1.4501        | 14.42 | 1160000 | 1.4385          |
  | 1.4484        | 14.67 | 1180000 | 1.4373          |
  | 1.449         | 14.92 | 1200000 | 1.4360          |
  
  
  ### Framework versions
  
  - Transformers 4.19.2
  - Pytorch 1.11.0
  - Datasets 2.2.2
  - Tokenizers 0.12.1
  