readme: ---
  license: mit
  tags:
  - generated_from_trainer
  model-index:
  - name: chile-gpt
    results: []
  ---
  
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
  should probably proofread and complete it, then remove this comment. -->
  
  # chile-gpt
  
  This model is a fine-tuned version of [DeepESP/gpt2-spanish](https://huggingface.co/DeepESP/gpt2-spanish) on an unknown dataset.
  It achieves the following results on the evaluation set:
  - Loss: 9.4320
  
  ## Model description
  
  More information needed
  
  ## Intended uses & limitations
  
  More information needed
  
  ## Training and evaluation data
  
  More information needed
  
  ## Training procedure
  
  ### Training hyperparameters
  
  The following hyperparameters were used during training:
  - learning_rate: 0.005
  - train_batch_size: 32
  - eval_batch_size: 32
  - seed: 42
  - gradient_accumulation_steps: 16
  - total_train_batch_size: 512
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  - lr_scheduler_type: cosine
  - lr_scheduler_warmup_steps: 1000
  - num_epochs: 50
  - mixed_precision_training: Native AMP
  
  ### Training results
  
  | Training Loss | Epoch | Step | Validation Loss |
  |:-------------:|:-----:|:----:|:---------------:|
  | 10.6676       | 0.98  | 6    | 9.5748          |
  | 9.6237        | 1.98  | 12   | 9.2470          |
  | 9.2815        | 2.98  | 18   | 8.8724          |
  | 8.8097        | 3.98  | 24   | 8.3629          |
  | 8.2296        | 4.98  | 30   | 7.8407          |
  | 7.6891        | 5.98  | 36   | 7.4161          |
  | 7.3013        | 6.98  | 42   | 7.1598          |
  | 7.0671        | 7.98  | 48   | 7.0080          |
  | 6.9404        | 8.98  | 54   | 6.9133          |
  | 6.7543        | 9.98  | 60   | 6.7723          |
  | 6.5845        | 10.98 | 66   | 6.6619          |
  | 6.4193        | 11.98 | 72   | 6.5965          |
  | 6.2554        | 12.98 | 78   | 6.5185          |
  | 6.0993        | 13.98 | 84   | 6.4632          |
  | 5.93          | 14.98 | 90   | 6.4155          |
  | 5.7684        | 15.98 | 96   | 6.4183          |
  | 5.6242        | 16.98 | 102  | 6.3981          |
  | 5.4577        | 17.98 | 108  | 6.4609          |
  | 5.2898        | 18.98 | 114  | 6.4577          |
  | 5.1113        | 19.98 | 120  | 6.5617          |
  | 4.9319        | 20.98 | 126  | 6.5827          |
  | 4.7464        | 21.98 | 132  | 6.6961          |
  | 4.5505        | 22.98 | 138  | 6.8359          |
  | 4.341         | 23.98 | 144  | 6.9193          |
  | 4.1324        | 24.98 | 150  | 7.0325          |
  | 3.8938        | 25.98 | 156  | 7.1993          |
  | 3.6691        | 26.98 | 162  | 7.3179          |
  | 3.4316        | 27.98 | 168  | 7.4708          |
  | 3.2041        | 28.98 | 174  | 7.5654          |
  | 2.9614        | 29.98 | 180  | 7.7535          |
  | 2.7189        | 30.98 | 186  | 7.8551          |
  | 2.4944        | 31.98 | 192  | 8.0094          |
  | 2.2624        | 32.98 | 198  | 8.0527          |
  | 2.0292        | 33.98 | 204  | 8.1857          |
  | 1.809         | 34.98 | 210  | 8.3468          |
  | 1.597         | 35.98 | 216  | 8.4307          |
  | 1.3849        | 36.98 | 222  | 8.6230          |
  | 1.2081        | 37.98 | 228  | 8.6666          |
  | 1.0273        | 38.98 | 234  | 8.7926          |
  | 0.8661        | 39.98 | 240  | 8.8861          |
  | 0.7308        | 40.98 | 246  | 8.9042          |
  | 0.6189        | 41.98 | 252  | 8.9202          |
  | 0.5335        | 42.98 | 258  | 9.0861          |
  | 0.459         | 43.98 | 264  | 9.1198          |
  | 0.3958        | 44.98 | 270  | 9.2129          |
  | 0.3587        | 45.98 | 276  | 9.2434          |
  | 0.3222        | 46.98 | 282  | 9.3005          |
  | 0.2948        | 47.98 | 288  | 9.3961          |
  | 0.2677        | 48.98 | 294  | 9.4605          |
  | 0.2348        | 49.98 | 300  | 9.4320          |
  
  
  ### Framework versions
  
  - Transformers 4.24.0
  - Pytorch 1.13.0+rocm5.2
  - Datasets 2.6.1
  - Tokenizers 0.13.2
  