readme: ---
  tags:
  - generated_from_trainer
  model-index:
  - name: enlm-roberta
    results: []
  ---
  
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
  should probably proofread and complete it, then remove this comment. -->
  
  # enlm-roberta
  
  This model is a fine-tuned version of [manirai91/enlm-roberta](https://huggingface.co/manirai91/enlm-roberta) on an unknown dataset.
  It achieves the following results on the evaluation set:
  - Loss: 1.4193
  
  ## Model description
  
  More information needed
  
  ## Intended uses & limitations
  
  More information needed
  
  ## Training and evaluation data
  
  More information needed
  
  ## Training procedure
  
  ### Training hyperparameters
  
  The following hyperparameters were used during training:
  - learning_rate: 6e-05
  - train_batch_size: 16
  - eval_batch_size: 16
  - seed: 42
  - distributed_type: multi-GPU
  - num_devices: 4
  - gradient_accumulation_steps: 128
  - total_train_batch_size: 8192
  - total_eval_batch_size: 64
  - optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-06
  - lr_scheduler_type: polynomial
  - num_epochs: 10
  
  ### Training results
  
  | Training Loss | Epoch | Step | Validation Loss |
  |:-------------:|:-----:|:----:|:---------------:|
  | 1.5984        | 0.13  | 160  | 1.4905          |
  | 1.6149        | 0.27  | 320  | 1.4969          |
  | 1.6285        | 0.61  | 480  | 1.5087          |
  | 1.6473        | 0.96  | 640  | 1.5173          |
  | 1.679         | 1.3   | 800  | 1.5441          |
  | 1.683         | 1.64  | 960  | 1.5576          |
  | 1.6942        | 1.99  | 1120 | 1.5681          |
  | 1.6921        | 2.33  | 1280 | 1.5659          |
  | 1.6986        | 2.67  | 1440 | 1.5684          |
  | 1.8496        | 3.02  | 1600 | 1.5586          |
  | 1.6807        | 3.36  | 1760 | 1.5503          |
  | 1.6744        | 3.7   | 1920 | 1.5466          |
  | 1.6838        | 4.05  | 2080 | 1.5471          |
  | 1.6725        | 4.39  | 2240 | 1.5371          |
  | 1.6663        | 4.73  | 2400 | 1.5433          |
  | 1.6644        | 5.08  | 2560 | 1.5321          |
  | 1.66          | 5.42  | 2720 | 1.5325          |
  | 1.6535        | 5.76  | 2880 | 1.5272          |
  | 1.651         | 6.11  | 3040 | 1.5253          |
  | 1.6432        | 6.45  | 3200 | 1.5207          |
  | 1.6452        | 6.79  | 3360 | 1.5239          |
  | 1.6398        | 7.14  | 3520 | 1.5168          |
  | 1.6308        | 7.48  | 3680 | 1.5088          |
  | 1.6332        | 7.82  | 3840 | 1.5065          |
  | 1.6261        | 8.17  | 4000 | 1.5024          |
  | 1.6194        | 8.51  | 4160 | 1.5085          |
  | 1.6178        | 8.85  | 4320 | 1.4961          |
  | 1.6137        | 9.2   | 4480 | 1.4949          |
  | 1.613         | 9.54  | 4640 | 1.4953          |
  | 1.6048        | 9.88  | 4800 | 1.4905          |
  | 1.6058        | 10.23 | 4960 | 1.4893          |
  | 1.6036        | 10.57 | 5120 | 1.4826          |
  | 1.5976        | 10.91 | 5280 | 1.4846          |
  | 1.593         | 11.26 | 5440 | 1.4792          |
  | 1.5914        | 11.6  | 5600 | 1.4734          |
  | 1.5863        | 11.94 | 5760 | 1.4731          |
  | 1.5828        | 12.29 | 5920 | 1.4702          |
  | 1.5831        | 12.63 | 6080 | 1.4649          |
  | 1.5796        | 12.97 | 6240 | 1.4611          |
  | 1.5717        | 13.32 | 6400 | 1.4580          |
  | 1.5737        | 13.66 | 6560 | 1.4576          |
  | 1.7137        | 14.0  | 6720 | 1.4571          |
  | 1.5651        | 14.35 | 6880 | 1.4543          |
  | 1.561         | 14.69 | 7040 | 1.4469          |
  | 1.5578        | 15.25 | 7200 | 1.4469          |
  | 1.5531        | 15.6  | 7360 | 1.4430          |
  | 1.5548        | 15.94 | 7520 | 1.4408          |
  | 1.5523        | 16.28 | 7680 | 1.4390          |
  | 1.5467        | 16.63 | 7840 | 1.4357          |
  | 1.5467        | 16.97 | 8000 | 1.4328          |
  | 1.5406        | 17.23 | 8160 | 1.4290          |
  | 1.5379        | 17.58 | 8320 | 1.4321          |
  | 1.5349        | 17.92 | 8480 | 1.4277          |
  | 1.5343        | 18.26 | 8640 | 1.4238          |
  | 1.5302        | 18.61 | 8800 | 1.4206          |
  | 1.5293        | 18.95 | 8960 | 1.4198          |
  | 1.5278        | 19.29 | 9120 | 1.4207          |
  | 1.523         | 19.64 | 9280 | 1.4193          |
  
  
  ### Framework versions
  
  - Transformers 4.20.1
  - Pytorch 1.11.0
  - Datasets 2.3.2
  - Tokenizers 0.12.1
  