readme: ---
  license: mit
  tags:
  - generated_from_trainer
  model-index:
  - name: borges-gpt-collab
    results: []
  ---
  
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
  should probably proofread and complete it, then remove this comment. -->
  
  # borges-gpt-collab
  
  This model is a fine-tuned version of [DeepESP/gpt2-spanish](https://huggingface.co/DeepESP/gpt2-spanish) on an unknown dataset.
  It achieves the following results on the evaluation set:
  - Loss: 8.3468
  
  ## Model description
  
  More information needed
  
  ## Intended uses & limitations
  
  More information needed
  
  ## Training and evaluation data
  
  More information needed
  
  ## Training procedure
  
  ### Training hyperparameters
  
  The following hyperparameters were used during training:
  - learning_rate: 0.0005
  - train_batch_size: 32
  - eval_batch_size: 32
  - seed: 42
  - gradient_accumulation_steps: 16
  - total_train_batch_size: 512
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  - lr_scheduler_type: cosine
  - lr_scheduler_warmup_steps: 500
  - num_epochs: 70
  - mixed_precision_training: Native AMP
  
  ### Training results
  
  | Training Loss | Epoch | Step | Validation Loss |
  |:-------------:|:-----:|:----:|:---------------:|
  | 11.2135       | 0.96  | 7    | 10.2022         |
  | 10.3195       | 1.96  | 14   | 9.6343          |
  | 9.9127        | 2.96  | 21   | 9.4637          |
  | 9.7295        | 3.96  | 28   | 9.2993          |
  | 9.527         | 4.96  | 35   | 9.0962          |
  | 9.2648        | 5.96  | 42   | 8.8294          |
  | 8.9309        | 6.96  | 49   | 8.5103          |
  | 8.5639        | 7.96  | 56   | 8.1858          |
  | 8.2034        | 8.96  | 63   | 7.8816          |
  | 7.8665        | 9.96  | 70   | 7.6303          |
  | 7.5715        | 10.96 | 77   | 7.4307          |
  | 7.3259        | 11.96 | 84   | 7.2632          |
  | 7.136         | 12.96 | 91   | 7.1494          |
  | 6.9558        | 13.96 | 98   | 7.0957          |
  | 6.8068        | 14.96 | 105  | 7.0199          |
  | 6.6656        | 15.96 | 112  | 6.9554          |
  | 6.5264        | 16.96 | 119  | 6.9324          |
  | 6.3843        | 17.96 | 126  | 6.8940          |
  | 6.2204        | 18.96 | 133  | 6.8799          |
  | 6.0915        | 19.96 | 140  | 6.8788          |
  | 5.9532        | 20.96 | 147  | 6.8719          |
  | 5.8169        | 21.96 | 154  | 6.8647          |
  | 5.6531        | 22.96 | 161  | 6.8865          |
  | 5.5125        | 23.96 | 168  | 6.8940          |
  | 5.3666        | 24.96 | 175  | 6.9248          |
  | 5.2377        | 25.96 | 182  | 6.9421          |
  | 5.1115        | 26.96 | 189  | 6.9631          |
  | 4.9639        | 27.96 | 196  | 7.0135          |
  | 4.824         | 28.96 | 203  | 7.0352          |
  | 4.6886        | 29.96 | 210  | 7.0729          |
  | 4.5538        | 30.96 | 217  | 7.1385          |
  | 4.4126        | 31.96 | 224  | 7.1561          |
  | 4.2486        | 32.96 | 231  | 7.1792          |
  | 4.0955        | 33.96 | 238  | 7.2767          |
  | 3.9333        | 34.96 | 245  | 7.2815          |
  | 3.7914        | 35.96 | 252  | 7.3463          |
  | 3.618         | 36.96 | 259  | 7.3864          |
  | 3.4453        | 37.96 | 266  | 7.4394          |
  | 3.2795        | 38.96 | 273  | 7.4730          |
  | 3.0994        | 39.96 | 280  | 7.4880          |
  | 2.9143        | 40.96 | 287  | 7.5567          |
  | 2.741         | 41.96 | 294  | 7.5451          |
  | 2.5698        | 42.96 | 301  | 7.5966          |
  | 2.3855        | 43.96 | 308  | 7.6898          |
  | 2.2059        | 44.96 | 315  | 7.6957          |
  | 2.0634        | 45.96 | 322  | 7.7503          |
  | 1.8719        | 46.96 | 329  | 7.8369          |
  | 1.7059        | 47.96 | 336  | 7.8411          |
  | 1.54          | 48.96 | 343  | 7.8316          |
  | 1.3768        | 49.96 | 350  | 7.8630          |
  | 1.2177        | 50.96 | 357  | 7.9360          |
  | 1.0663        | 51.96 | 364  | 7.9886          |
  | 0.9569        | 52.96 | 371  | 8.0187          |
  | 0.8281        | 53.96 | 378  | 8.0274          |
  | 0.7074        | 54.96 | 385  | 8.1010          |
  | 0.6095        | 55.96 | 392  | 8.1594          |
  | 0.5262        | 56.96 | 399  | 8.1010          |
  | 0.4678        | 57.96 | 406  | 8.1440          |
  | 0.4105        | 58.96 | 413  | 8.1638          |
  | 0.3766        | 59.96 | 420  | 8.1534          |
  | 0.3425        | 60.96 | 427  | 8.1980          |
  | 0.321         | 61.96 | 434  | 8.2184          |
  | 0.3061        | 62.96 | 441  | 8.2499          |
  | 0.2852        | 63.96 | 448  | 8.1690          |
  | 0.2698        | 64.96 | 455  | 8.2160          |
  | 0.2628        | 65.96 | 462  | 8.2616          |
  | 0.2619        | 66.96 | 469  | 8.2948          |
  | 0.2544        | 67.96 | 476  | 8.3553          |
  | 0.2414        | 68.96 | 483  | 8.3712          |
  | 0.2177        | 69.96 | 490  | 8.3468          |
  
  
  ### Framework versions
  
  - Transformers 4.24.0
  - Pytorch 1.13.0+rocm5.2
  - Datasets 2.6.1
  - Tokenizers 0.13.2
  