readme: ---
  license: apache-2.0
  tags:
  - generated_from_trainer
  model-index:
  - name: neuroscience-to-dev-bio
    results: []
  ---
  
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
  should probably proofread and complete it, then remove this comment. -->
  
  # neuroscience-to-dev-bio
  
  This model is a fine-tuned version of [facebook/bart-large](https://huggingface.co/facebook/bart-large) on an unknown dataset.
  It achieves the following results on the evaluation set:
  - Loss: 0.0374
  
  ## Model description
  
  More information needed
  
  ## Intended uses & limitations
  
  More information needed
  
  ## Training and evaluation data
  
  More information needed
  
  ## Training procedure
  
  ### Training hyperparameters
  
  The following hyperparameters were used during training:
  - learning_rate: 5e-05
  - train_batch_size: 1
  - eval_batch_size: 1
  - seed: 42
  - gradient_accumulation_steps: 128
  - total_train_batch_size: 128
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  - lr_scheduler_type: linear
  - lr_scheduler_warmup_steps: 100
  - num_epochs: 5000
  - mixed_precision_training: Native AMP
  
  ### Training results
  
  | Training Loss | Epoch | Step | Validation Loss |
  |:-------------:|:-----:|:----:|:---------------:|
  | 19.3369       | 0.95  | 7    | 17.9500         |
  | 17.1316       | 1.95  | 14   | 15.1026         |
  | 14.0654       | 2.95  | 21   | 12.5013         |
  | 12.6374       | 3.95  | 28   | 11.3803         |
  | 11.7608       | 4.95  | 35   | 10.2692         |
  | 10.5271       | 5.95  | 42   | 8.7652          |
  | 9.0429        | 6.95  | 49   | 6.8763          |
  | 7.4963        | 7.95  | 56   | 5.9052          |
  | 6.6044        | 8.95  | 63   | 5.3443          |
  | 6.007         | 9.95  | 70   | 4.8687          |
  | 5.4706        | 10.95 | 77   | 4.3708          |
  | 4.8812        | 11.95 | 84   | 3.8094          |
  | 4.2359        | 12.95 | 91   | 3.1743          |
  | 3.4727        | 13.95 | 98   | 2.4480          |
  | 2.6582        | 14.95 | 105  | 1.6751          |
  | 1.8084        | 15.95 | 112  | 0.9828          |
  | 1.0742        | 16.95 | 119  | 0.5074          |
  | 0.5521        | 17.95 | 126  | 0.2471          |
  | 0.263         | 18.95 | 133  | 0.1276          |
  | 0.1281        | 19.95 | 140  | 0.0761          |
  | 0.0826        | 20.95 | 147  | 0.0620          |
  | 0.0419        | 21.95 | 154  | 0.0434          |
  | 0.0685        | 22.95 | 161  | 0.1522          |
  | 0.1332        | 23.95 | 168  | 0.0536          |
  | 0.0405        | 24.95 | 175  | 0.0405          |
  | 0.0214        | 25.95 | 182  | 0.0380          |
  | 0.0142        | 26.95 | 189  | 0.0370          |
  | 0.0202        | 27.95 | 196  | 0.0375          |
  | 0.0105        | 28.95 | 203  | 0.0413          |
  | 0.0092        | 29.95 | 210  | 0.0370          |
  | 0.0083        | 30.95 | 217  | 0.0384          |
  | 0.0079        | 31.95 | 224  | 0.0406          |
  | 0.0381        | 32.95 | 231  | 0.0371          |
  | 0.011         | 33.95 | 238  | 0.0439          |
  | 0.0066        | 34.95 | 245  | 0.0374          |
  
  
  ### Framework versions
  
  - Transformers 4.21.2
  - Pytorch 1.12.1+cu113
  - Datasets 2.4.0
  - Tokenizers 0.12.1
  