readme: ---
  license: mit
  tags:
  - generated_from_trainer
  model-index:
  - name: BERiT_2000_custom_architecture_2
    results: []
  ---
  
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
  should probably proofread and complete it, then remove this comment. -->
  
  # BERiT_2000_custom_architecture_2
  
  This model is a fine-tuned version of [roberta-base](https://huggingface.co/roberta-base) on an unknown dataset.
  It achieves the following results on the evaluation set:
  - Loss: 5.9854
  
  ## Model description
  
  More information needed
  
  ## Intended uses & limitations
  
  More information needed
  
  ## Training and evaluation data
  
  More information needed
  
  ## Training procedure
  
  ### Training hyperparameters
  
  The following hyperparameters were used during training:
  - learning_rate: 0.0005
  - train_batch_size: 8
  - eval_batch_size: 8
  - seed: 42
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  - lr_scheduler_type: linear
  - num_epochs: 20
  
  ### Training results
  
  | Training Loss | Epoch | Step  | Validation Loss |
  |:-------------:|:-----:|:-----:|:---------------:|
  | 16.4316       | 0.19  | 500   | 9.0685          |
  | 8.2958        | 0.39  | 1000  | 7.6483          |
  | 7.4324        | 0.58  | 1500  | 7.1707          |
  | 7.0054        | 0.77  | 2000  | 6.8592          |
  | 6.8522        | 0.97  | 2500  | 6.7710          |
  | 6.7538        | 1.16  | 3000  | 6.5845          |
  | 6.634         | 1.36  | 3500  | 6.4525          |
  | 6.5784        | 1.55  | 4000  | 6.3129          |
  | 6.5135        | 1.74  | 4500  | 6.3312          |
  | 6.4552        | 1.94  | 5000  | 6.2546          |
  | 6.4685        | 2.13  | 5500  | 6.2857          |
  | 6.4356        | 2.32  | 6000  | 6.2285          |
  | 6.3566        | 2.52  | 6500  | 6.2295          |
  | 6.394         | 2.71  | 7000  | 6.1790          |
  | 6.3412        | 2.9   | 7500  | 6.1880          |
  | 6.3115        | 3.1   | 8000  | 6.2130          |
  | 6.3163        | 3.29  | 8500  | 6.1831          |
  | 6.2978        | 3.49  | 9000  | 6.1945          |
  | 6.3082        | 3.68  | 9500  | 6.1485          |
  | 6.2729        | 3.87  | 10000 | 6.1752          |
  | 6.307         | 4.07  | 10500 | 6.1331          |
  | 6.2494        | 4.26  | 11000 | 6.1082          |
  | 6.2523        | 4.45  | 11500 | 6.2110          |
  | 6.2455        | 4.65  | 12000 | 6.1326          |
  | 6.2399        | 4.84  | 12500 | 6.1779          |
  | 6.2297        | 5.03  | 13000 | 6.1587          |
  | 6.2374        | 5.23  | 13500 | 6.1458          |
  | 6.2265        | 5.42  | 14000 | 6.1370          |
  | 6.2222        | 5.62  | 14500 | 6.1511          |
  | 6.2209        | 5.81  | 15000 | 6.1320          |
  | 6.2146        | 6.0   | 15500 | 6.1124          |
  | 6.214         | 6.2   | 16000 | 6.1439          |
  | 6.1907        | 6.39  | 16500 | 6.0981          |
  | 6.2119        | 6.58  | 17000 | 6.1465          |
  | 6.1858        | 6.78  | 17500 | 6.1594          |
  | 6.1552        | 6.97  | 18000 | 6.0742          |
  | 6.1926        | 7.16  | 18500 | 6.1176          |
  | 6.1813        | 7.36  | 19000 | 6.0107          |
  | 6.1812        | 7.55  | 19500 | 6.0852          |
  | 6.1852        | 7.75  | 20000 | 6.0845          |
  | 6.1945        | 7.94  | 20500 | 6.1260          |
  | 6.1542        | 8.13  | 21000 | 6.1032          |
  | 6.1685        | 8.33  | 21500 | 6.0650          |
  | 6.1619        | 8.52  | 22000 | 6.1028          |
  | 6.1279        | 8.71  | 22500 | 6.1269          |
  | 6.1575        | 8.91  | 23000 | 6.0793          |
  | 6.1401        | 9.1   | 23500 | 6.1479          |
  | 6.159         | 9.3   | 24000 | 6.0319          |
  | 6.1227        | 9.49  | 24500 | 6.0677          |
  | 6.1201        | 9.68  | 25000 | 6.0527          |
  | 6.1473        | 9.88  | 25500 | 6.1305          |
  | 6.1539        | 10.07 | 26000 | 6.1079          |
  | 6.091         | 10.26 | 26500 | 6.1219          |
  | 6.1015        | 10.46 | 27000 | 6.1317          |
  | 6.1048        | 10.65 | 27500 | 6.1149          |
  | 6.0955        | 10.84 | 28000 | 6.1216          |
  | 6.129         | 11.04 | 28500 | 6.0427          |
  | 6.1007        | 11.23 | 29000 | 6.1289          |
  | 6.1266        | 11.43 | 29500 | 6.0564          |
  | 6.1203        | 11.62 | 30000 | 6.1143          |
  | 6.1038        | 11.81 | 30500 | 6.0957          |
  | 6.0989        | 12.01 | 31000 | 6.0707          |
  | 6.0571        | 12.2  | 31500 | 6.0013          |
  | 6.1017        | 12.39 | 32000 | 6.1356          |
  | 6.0649        | 12.59 | 32500 | 6.0981          |
  | 6.0704        | 12.78 | 33000 | 6.0588          |
  | 6.088         | 12.97 | 33500 | 6.0796          |
  | 6.1112        | 13.17 | 34000 | 6.0809          |
  | 6.0888        | 13.36 | 34500 | 6.0776          |
  | 6.0482        | 13.56 | 35000 | 6.0710          |
  | 6.0588        | 13.75 | 35500 | 6.0877          |
  | 6.0517        | 13.94 | 36000 | 6.0650          |
  | 6.0832        | 14.14 | 36500 | 5.9890          |
  | 6.0655        | 14.33 | 37000 | 6.0445          |
  | 6.0705        | 14.52 | 37500 | 6.0037