readme: ---
  tags:
  - generated_from_trainer
  model-index:
  - name: dna_bert_3_1000seq-finetuned
    results: []
  ---
  
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
  should probably proofread and complete it, then remove this comment. -->
  
  # dna_bert_3_1000seq-finetuned
  
  This model is a fine-tuned version of [armheb/DNA_bert_3](https://huggingface.co/armheb/DNA_bert_3) on an unknown dataset.
  It achieves the following results on the evaluation set:
  - Loss: 0.4684
  
  ## Model description
  
  More information needed
  
  ## Intended uses & limitations
  
  More information needed
  
  ## Training and evaluation data
  
  More information needed
  
  ## Training procedure
  
  ### Training hyperparameters
  
  The following hyperparameters were used during training:
  - learning_rate: 2e-05
  - train_batch_size: 16
  - eval_batch_size: 16
  - seed: 42
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  - lr_scheduler_type: linear
  - num_epochs: 100
  - mixed_precision_training: Native AMP
  
  ### Training results
  
  | Training Loss | Epoch | Step | Validation Loss |
  |:-------------:|:-----:|:----:|:---------------:|
  | 0.8607        | 1.0   | 62   | 0.6257          |
  | 0.6177        | 2.0   | 124  | 0.6120          |
  | 0.6098        | 3.0   | 186  | 0.6062          |
  | 0.604         | 4.0   | 248  | 0.6052          |
  | 0.5999        | 5.0   | 310  | 0.6040          |
  | 0.5982        | 6.0   | 372  | 0.5996          |
  | 0.5985        | 7.0   | 434  | 0.5985          |
  | 0.5956        | 8.0   | 496  | 0.5968          |
  | 0.5936        | 9.0   | 558  | 0.5950          |
  | 0.5908        | 10.0  | 620  | 0.5941          |
  | 0.5904        | 11.0  | 682  | 0.5932          |
  | 0.59          | 12.0  | 744  | 0.5917          |
  | 0.5877        | 13.0  | 806  | 0.5921          |
  | 0.5847        | 14.0  | 868  | 0.5903          |
  | 0.5831        | 15.0  | 930  | 0.5887          |
  | 0.5852        | 16.0  | 992  | 0.5878          |
  | 0.5805        | 17.0  | 1054 | 0.5872          |
  | 0.5795        | 18.0  | 1116 | 0.5853          |
  | 0.5754        | 19.0  | 1178 | 0.5869          |
  | 0.5757        | 20.0  | 1240 | 0.5839          |
  | 0.5722        | 21.0  | 1302 | 0.5831          |
  | 0.5693        | 22.0  | 1364 | 0.5811          |
  | 0.5667        | 23.0  | 1426 | 0.5802          |
  | 0.5652        | 24.0  | 1488 | 0.5775          |
  | 0.5608        | 25.0  | 1550 | 0.5788          |
  | 0.5591        | 26.0  | 1612 | 0.5724          |
  | 0.5538        | 27.0  | 1674 | 0.5736          |
  | 0.552         | 28.0  | 1736 | 0.5689          |
  | 0.5483        | 29.0  | 1798 | 0.5689          |
  | 0.5442        | 30.0  | 1860 | 0.5671          |
  | 0.5405        | 31.0  | 1922 | 0.5658          |
  | 0.537         | 32.0  | 1984 | 0.5605          |
  | 0.5349        | 33.0  | 2046 | 0.5575          |
  | 0.5275        | 34.0  | 2108 | 0.5569          |
  | 0.5227        | 35.0  | 2170 | 0.5537          |
  | 0.52          | 36.0  | 2232 | 0.5509          |
  | 0.5173        | 37.0  | 2294 | 0.5504          |
  | 0.5123        | 38.0  | 2356 | 0.5435          |
  | 0.5088        | 39.0  | 2418 | 0.5472          |
  | 0.5037        | 40.0  | 2480 | 0.5383          |
  | 0.501         | 41.0  | 2542 | 0.5379          |
  | 0.4931        | 42.0  | 2604 | 0.5365          |
  | 0.4923        | 43.0  | 2666 | 0.5328          |
  | 0.4879        | 44.0  | 2728 | 0.5301          |
  | 0.482         | 45.0  | 2790 | 0.5295          |
  | 0.4805        | 46.0  | 2852 | 0.5261          |
  | 0.4772        | 47.0  | 2914 | 0.5221          |
  | 0.4738        | 48.0  | 2976 | 0.5234          |
  | 0.4674        | 49.0  | 3038 | 0.5210          |
  | 0.4646        | 50.0  | 3100 | 0.5169          |
  | 0.4621        | 51.0  | 3162 | 0.5142          |
  | 0.4574        | 52.0  | 3224 | 0.5129          |
  | 0.4552        | 53.0  | 3286 | 0.5127          |
  | 0.4539        | 54.0  | 3348 | 0.5124          |
  | 0.4506        | 55.0  | 3410 | 0.5076          |
  | 0.4457        | 56.0  | 3472 | 0.5082          |
  | 0.4454        | 57.0  | 3534 | 0.5027          |
  | 0.4398        | 58.0  | 3596 | 0.5019          |
  | 0.4386        | 59.0  | 3658 | 0.4998          |
  | 0.4332        | 60.0  | 3720 | 0.4970          |
  | 0.4277        | 61.0  | 3782 | 0.4995          |
  | 0.4273        | 62.0  | 3844 | 0.4962          |
  | 0.4235        | 63.0  | 3906 | 0.4909          |
  | 0.4201        | 64.0  | 3968 | 0.4913          |
  | 0.4198        | 65.0  | 4030 | 0.4899          |
  | 0.4182        | 66.0  | 4092 | 0.4919          |
  | 0.4157        | 67.0  | 4154 | 0.4902          |
  | 0.4104        | 68.0  | 4216 | 0.4881          |
  | 0.4095        | 69.0  | 4278 | 0.4881          |
  | 0.4077        | 70.0  | 4340 | 0.4861          |
  | 0.4064        | 71.0  | 4402 | 0.4868          |
  | 0.4041        | 72.0  | 4464 | 0.4826          |
  | 0.4029        | 73.0  | 4526 | 0.4833          |
  | 0.3976        | 74.0  | 4588 | 0.4819          |
  | 0.3997        | 75.0  | 4650 | 0.4809          |
  | 0.3974        | 76.0  | 4712 | 0.