readme: ---
  license: mit
  tags:
  - generated_from_trainer
  model-index:
  - name: Romance-baseline
    results: []
  ---
  
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
  should probably proofread and complete it, then remove this comment. -->
  
  # Romance-baseline
  
  This model is a fine-tuned version of [gpt2](https://huggingface.co/gpt2) on the None dataset.
  It achieves the following results on the evaluation set:
  - Loss: 6.5909
  
  ## Model description
  
  More information needed
  
  ## Intended uses & limitations
  
  More information needed
  
  ## Training and evaluation data
  
  More information needed
  
  ## Training procedure
  
  ### Training hyperparameters
  
  The following hyperparameters were used during training:
  - learning_rate: 5e-05
  - train_batch_size: 32
  - eval_batch_size: 32
  - seed: 42
  - gradient_accumulation_steps: 8
  - total_train_batch_size: 256
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  - lr_scheduler_type: cosine
  - lr_scheduler_warmup_steps: 1000
  - num_epochs: 50
  - mixed_precision_training: Native AMP
  
  ### Training results
  
  | Training Loss | Epoch | Step | Validation Loss |
  |:-------------:|:-----:|:----:|:---------------:|
  | No log        | 0.94  | 15   | 10.7009         |
  | No log        | 1.94  | 30   | 10.0799         |
  | No log        | 2.94  | 45   | 9.6627          |
  | No log        | 3.94  | 60   | 9.4619          |
  | No log        | 4.94  | 75   | 9.2970          |
  | No log        | 5.94  | 90   | 9.0919          |
  | No log        | 6.94  | 105  | 8.9071          |
  | No log        | 7.94  | 120  | 8.7240          |
  | No log        | 8.94  | 135  | 8.5485          |
  | No log        | 9.94  | 150  | 8.3952          |
  | No log        | 10.94 | 165  | 8.2469          |
  | No log        | 11.94 | 180  | 8.1193          |
  | No log        | 12.94 | 195  | 7.9918          |
  | No log        | 13.94 | 210  | 7.8662          |
  | No log        | 14.94 | 225  | 7.7394          |
  | No log        | 15.94 | 240  | 7.6219          |
  | No log        | 16.94 | 255  | 7.5135          |
  | No log        | 17.94 | 270  | 7.4110          |
  | No log        | 18.94 | 285  | 7.3021          |
  | No log        | 19.94 | 300  | 7.2021          |
  | No log        | 20.94 | 315  | 7.1276          |
  | No log        | 21.94 | 330  | 7.0278          |
  | No log        | 22.94 | 345  | 6.9627          |
  | No log        | 23.94 | 360  | 6.8806          |
  | No log        | 24.94 | 375  | 6.8214          |
  | No log        | 25.94 | 390  | 6.7725          |
  | No log        | 26.94 | 405  | 6.7101          |
  | No log        | 27.94 | 420  | 6.6792          |
  | No log        | 28.94 | 435  | 6.6361          |
  | No log        | 29.94 | 450  | 6.5950          |
  | No log        | 30.94 | 465  | 6.5745          |
  | No log        | 31.94 | 480  | 6.5469          |
  | No log        | 32.94 | 495  | 6.5520          |
  | No log        | 33.94 | 510  | 6.5121          |
  | No log        | 34.94 | 525  | 6.5255          |
  | No log        | 35.94 | 540  | 6.5179          |
  | No log        | 36.94 | 555  | 6.5079          |
  | No log        | 37.94 | 570  | 6.5138          |
  | No log        | 38.94 | 585  | 6.5170          |
  | No log        | 39.94 | 600  | 6.4807          |
  | No log        | 40.94 | 615  | 6.5338          |
  | No log        | 41.94 | 630  | 6.4960          |
  | No log        | 42.94 | 645  | 6.5342          |
  | No log        | 43.94 | 660  | 6.5119          |
  | No log        | 44.94 | 675  | 6.5614          |
  | No log        | 45.94 | 690  | 6.5235          |
  | No log        | 46.94 | 705  | 6.5388          |
  | No log        | 47.94 | 720  | 6.5574          |
  | No log        | 48.94 | 735  | 6.5581          |
  | No log        | 49.94 | 750  | 6.5909          |
  
  
  ### Framework versions
  
  - Transformers 4.24.0
  - Pytorch 1.12.1+cu113
  - Datasets 2.6.1
  - Tokenizers 0.13.1
  