readme: ---
  tags:
  - generated_from_trainer
  datasets:
  - wikitext
  model-index:
  - name: msft-regular-model
    results: []
  ---
  
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
  should probably proofread and complete it, then remove this comment. -->
  
  # msft-regular-model
  
  This model is a fine-tuned version of [](https://huggingface.co/) on the wikitext dataset.
  It achieves the following results on the evaluation set:
  - Loss: 5.3420
  
  ## Model description
  
  More information needed
  
  ## Intended uses & limitations
  
  More information needed
  
  ## Training and evaluation data
  
  More information needed
  
  ## Training procedure
  
  ### Training hyperparameters
  
  The following hyperparameters were used during training:
  - learning_rate: 5e-05
  - train_batch_size: 16
  - eval_batch_size: 16
  - seed: 42
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  - lr_scheduler_type: linear
  - num_epochs: 20
  
  ### Training results
  
  | Training Loss | Epoch | Step  | Validation Loss |
  |:-------------:|:-----:|:-----:|:---------------:|
  | 9.1224        | 0.17  | 200   | 8.0736          |
  | 7.5229        | 0.34  | 400   | 7.1536          |
  | 7.0122        | 0.51  | 600   | 6.9072          |
  | 6.8296        | 0.69  | 800   | 6.7582          |
  | 6.709         | 0.86  | 1000  | 6.6436          |
  | 6.5882        | 1.03  | 1200  | 6.5563          |
  | 6.4807        | 1.2   | 1400  | 6.4784          |
  | 6.4172        | 1.37  | 1600  | 6.4165          |
  | 6.3403        | 1.54  | 1800  | 6.3555          |
  | 6.2969        | 1.71  | 2000  | 6.3107          |
  | 6.2346        | 1.89  | 2200  | 6.2691          |
  | 6.1767        | 2.06  | 2400  | 6.2299          |
  | 6.1326        | 2.23  | 2600  | 6.1937          |
  | 6.1035        | 2.4   | 2800  | 6.1602          |
  | 6.0624        | 2.57  | 3000  | 6.1241          |
  | 6.0393        | 2.74  | 3200  | 6.0971          |
  | 5.9982        | 2.91  | 3400  | 6.0656          |
  | 5.9526        | 3.08  | 3600  | 6.0397          |
  | 5.9086        | 3.26  | 3800  | 6.0104          |
  | 5.8922        | 3.43  | 4000  | 5.9888          |
  | 5.8631        | 3.6   | 4200  | 5.9661          |
  | 5.8396        | 3.77  | 4400  | 5.9407          |
  | 5.8055        | 3.94  | 4600  | 5.9177          |
  | 5.7763        | 4.11  | 4800  | 5.9007          |
  | 5.7314        | 4.28  | 5000  | 5.8834          |
  | 5.7302        | 4.46  | 5200  | 5.8620          |
  | 5.6987        | 4.63  | 5400  | 5.8451          |
  | 5.6754        | 4.8   | 5600  | 5.8242          |
  | 5.6571        | 4.97  | 5800  | 5.8059          |
  | 5.615         | 5.14  | 6000  | 5.7871          |
  | 5.596         | 5.31  | 6200  | 5.7817          |
  | 5.5738        | 5.48  | 6400  | 5.7570          |
  | 5.5641        | 5.66  | 6600  | 5.7431          |
  | 5.5503        | 5.83  | 6800  | 5.7271          |
  | 5.5214        | 6.0   | 7000  | 5.7108          |
  | 5.4712        | 6.17  | 7200  | 5.7018          |
  | 5.48          | 6.34  | 7400  | 5.6936          |
  | 5.4527        | 6.51  | 7600  | 5.6812          |
  | 5.4514        | 6.68  | 7800  | 5.6669          |
  | 5.4454        | 6.86  | 8000  | 5.6509          |
  | 5.399         | 7.03  | 8200  | 5.6408          |
  | 5.3747        | 7.2   | 8400  | 5.6327          |
  | 5.3667        | 7.37  | 8600  | 5.6197          |
  | 5.3652        | 7.54  | 8800  | 5.6084          |
  | 5.3394        | 7.71  | 9000  | 5.5968          |
  | 5.3349        | 7.88  | 9200  | 5.5870          |
  | 5.2994        | 8.05  | 9400  | 5.5826          |
  | 5.2793        | 8.23  | 9600  | 5.5710          |
  | 5.2716        | 8.4   | 9800  | 5.5623          |
  | 5.275         | 8.57  | 10000 | 5.5492          |
  | 5.264         | 8.74  | 10200 | 5.5449          |
  | 5.241         | 8.91  | 10400 | 5.5322          |
  | 5.2285        | 9.08  | 10600 | 5.5267          |
  | 5.2021        | 9.25  | 10800 | 5.5187          |
  | 5.1934        | 9.43  | 11000 | 5.5158          |
  | 5.1737        | 9.6   | 11200 | 5.5044          |
  | 5.1774        | 9.77  | 11400 | 5.5008          |
  | 5.1841        | 9.94  | 11600 | 5.4960          |
  | 5.1414        | 10.11 | 11800 | 5.4895          |
  | 5.1491        | 10.28 | 12000 | 5.4849          |
  | 5.1184        | 10.45 | 12200 | 5.4738          |
  | 5.1136        | 10.63 | 12400 | 5.4690          |
  | 5.1199        | 10.8  | 12600 | 5.4598          |
  | 5.1056        | 10.97 | 12800 | 5.4536          |
  | 5.0648        | 11.14 | 13000 | 5.4496          |
  | 5.0598        | 11.31 | 13200 | 5.4449          |
  | 5.0656        | 11.48 | 13400 | 5.4422          |
  | 5.0664        | 11.65 | 13600 | 5.4367          |
  | 5.0675        | 11.83 | 13800 | 5.4286          |
  | 5.0459        | 12.0  | 14000 | 5.4249          |
  | 5.0073        | 12.17 | 14200 | 5.4260          |
  | 5.0229        | 12.34 | 14400 | 5.4175          |
  | 5.0079        | 12.51 | 14600 | 5.4119          |
  | 5.0           | 12.68 | 14800 | 5.4194          |
  | 5.0094        | 12.85 | 15000 | 5.4068          |
  | 4.9967        | 13.02 | 152