readme: ---
  tags:
  - generated_from_trainer
  model-index:
  - name: from_scratch
    results: []
  ---
  
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
  should probably proofread and complete it, then remove this comment. -->
  
  # from_scratch
  
  This model is a fine-tuned version of [tokenizer/config.json](https://huggingface.co/tokenizer/config.json) on the None dataset.
  It achieves the following results on the evaluation set:
  - Loss: 0.4744
  
  ## Model description
  
  More information needed
  
  ## Intended uses & limitations
  
  More information needed
  
  ## Training and evaluation data
  
  More information needed
  
  ## Training procedure
  
  ### Training hyperparameters
  
  The following hyperparameters were used during training:
  - learning_rate: 0.0001
  - train_batch_size: 360
  - eval_batch_size: 360
  - seed: 42
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-06
  - lr_scheduler_type: linear
  - lr_scheduler_warmup_steps: 10000
  - num_epochs: 3.0
  - mixed_precision_training: Native AMP
  
  ### Training results
  
  | Training Loss | Epoch | Step    | Validation Loss |
  |:-------------:|:-----:|:-------:|:---------------:|
  | 1.0952        | 0.05  | 20000   | 1.0383          |
  | 0.936         | 0.1   | 40000   | 0.8852          |
  | 0.8679        | 0.14  | 60000   | 0.8207          |
  | 0.8276        | 0.19  | 80000   | 0.7796          |
  | 0.796         | 0.24  | 100000  | 0.7519          |
  | 0.7756        | 0.29  | 120000  | 0.7299          |
  | 0.7545        | 0.33  | 140000  | 0.7103          |
  | 0.7395        | 0.38  | 160000  | 0.6947          |
  | 0.7236        | 0.43  | 180000  | 0.6809          |
  | 0.7143        | 0.48  | 200000  | 0.6705          |
  | 0.705         | 0.52  | 220000  | 0.6585          |
  | 0.6904        | 0.57  | 240000  | 0.6479          |
  | 0.6835        | 0.62  | 260000  | 0.6388          |
  | 0.672         | 0.67  | 280000  | 0.6290          |
  | 0.665         | 0.72  | 300000  | 0.6217          |
  | 0.6581        | 0.76  | 320000  | 0.6136          |
  | 0.6466        | 0.81  | 340000  | 0.6071          |
  | 0.6396        | 0.86  | 360000  | 0.6000          |
  | 0.6343        | 0.91  | 380000  | 0.5940          |
  | 0.6286        | 0.95  | 400000  | 0.5880          |
  | 0.6183        | 1.0   | 420000  | 0.5809          |
  | 0.6134        | 1.05  | 440000  | 0.5757          |
  | 0.6094        | 1.1   | 460000  | 0.5693          |
  | 0.6032        | 1.15  | 480000  | 0.5641          |
  | 0.5954        | 1.19  | 500000  | 0.5596          |
  | 0.5915        | 1.24  | 520000  | 0.5532          |
  | 0.5845        | 1.29  | 540000  | 0.5489          |
  | 0.5823        | 1.34  | 560000  | 0.5437          |
  | 0.5754        | 1.38  | 580000  | 0.5393          |
  | 0.573         | 1.43  | 600000  | 0.5345          |
  | 0.5643        | 1.48  | 620000  | 0.5309          |
  | 0.5627        | 1.53  | 640000  | 0.5262          |
  | 0.56          | 1.57  | 660000  | 0.5220          |
  | 0.5554        | 1.62  | 680000  | 0.5186          |
  | 0.5507        | 1.67  | 700000  | 0.5152          |
  | 0.5494        | 1.72  | 720000  | 0.5117          |
  | 0.5445        | 1.77  | 740000  | 0.5076          |
  | 0.5396        | 1.81  | 760000  | 0.5051          |
  | 0.5363        | 1.86  | 780000  | 0.5026          |
  | 0.5356        | 1.91  | 800000  | 0.4998          |
  | 0.5303        | 1.96  | 820000  | 0.4982          |
  | 0.5583        | 2.0   | 840000  | 0.5195          |
  | 0.5565        | 2.05  | 860000  | 0.5180          |
  | 0.5535        | 2.1   | 880000  | 0.5158          |
  | 0.5497        | 2.15  | 900000  | 0.5133          |
  | 0.5511        | 2.19  | 920000  | 0.5110          |
  | 0.5439        | 2.24  | 940000  | 0.5085          |
  | 0.5413        | 2.29  | 960000  | 0.5060          |
  | 0.5376        | 2.34  | 980000  | 0.5023          |
  | 0.5333        | 2.39  | 1000000 | 0.5004          |
  | 0.5322        | 2.43  | 1020000 | 0.4973          |
  | 0.5312        | 2.48  | 1040000 | 0.4941          |
  | 0.5281        | 2.53  | 1060000 | 0.4921          |
  | 0.5267        | 2.58  | 1080000 | 0.4902          |
  | 0.5257        | 2.62  | 1100000 | 0.4871          |
  | 0.5174        | 2.67  | 1120000 | 0.4849          |
  | 0.5183        | 2.72  | 1140000 | 0.4825          |
  | 0.5181        | 2.77  | 1160000 | 0.4807          |
  | 0.5116        | 2.81  | 1180000 | 0.4784          |
  | 0.5092        | 2.86  | 1200000 | 0.4769          |
  | 0.5109        | 2.91  | 1220000 | 0.4757          |
  | 0.5102        | 2.96  | 1240000 | 0.4739          |
  
  
  ### Framework versions
  
  - Transformers 4.25.1
  - Pytorch 1.13.1+cu117
  - Datasets 2.8.0
  - Tokenizers 0.13.2
  