readme: ---
  license: mit
  tags:
  - generated_from_trainer
  model-index:
  - name: BERiT_2000_custom_architecture_3
    results: []
  ---
  
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
  should probably proofread and complete it, then remove this comment. -->
  
  # BERiT_2000_custom_architecture_3
  
  This model is a fine-tuned version of [roberta-base](https://huggingface.co/roberta-base) on an unknown dataset.
  It achieves the following results on the evaluation set:
  - Loss: 5.6575
  
  ## Model description
  
  More information needed
  
  ## Intended uses & limitations
  
  More information needed
  
  ## Training and evaluation data
  
  More information needed
  
  ## Training procedure
  
  ### Training hyperparameters
  
  The following hyperparameters were used during training:
  - learning_rate: 0.0005
  - train_batch_size: 8
  - eval_batch_size: 8
  - seed: 42
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  - lr_scheduler_type: linear
  - num_epochs: 40
  
  ### Training results
  
  | Training Loss | Epoch | Step   | Validation Loss |
  |:-------------:|:-----:|:------:|:---------------:|
  | 16.5165       | 0.19  | 500    | 8.9072          |
  | 8.208         | 0.39  | 1000   | 7.5024          |
  | 7.3849        | 0.58  | 1500   | 7.1180          |
  | 7.0298        | 0.77  | 2000   | 6.8964          |
  | 6.9022        | 0.97  | 2500   | 6.7857          |
  | 6.7756        | 1.16  | 3000   | 6.5215          |
  | 6.6462        | 1.36  | 3500   | 6.4494          |
  | 6.5787        | 1.55  | 4000   | 6.3246          |
  | 6.5193        | 1.74  | 4500   | 6.3231          |
  | 6.4626        | 1.94  | 5000   | 6.2818          |
  | 6.474         | 2.13  | 5500   | 6.3444          |
  | 6.4314        | 2.32  | 6000   | 6.2374          |
  | 6.3658        | 2.52  | 6500   | 6.2517          |
  | 6.4031        | 2.71  | 7000   | 6.2055          |
  | 6.3549        | 2.9   | 7500   | 6.2022          |
  | 6.3202        | 3.1   | 8000   | 6.2163          |
  | 6.3294        | 3.29  | 8500   | 6.2001          |
  | 6.2981        | 3.49  | 9000   | 6.1819          |
  | 6.3281        | 3.68  | 9500   | 6.1564          |
  | 6.2914        | 3.87  | 10000  | 6.2122          |
  | 6.3275        | 4.07  | 10500  | 6.1463          |
  | 6.2637        | 4.26  | 11000  | 6.1404          |
  | 6.2641        | 4.45  | 11500  | 6.2438          |
  | 6.2557        | 4.65  | 12000  | 6.1504          |
  | 6.2541        | 4.84  | 12500  | 6.1816          |
  | 6.2465        | 5.03  | 13000  | 6.1646          |
  | 6.2436        | 5.23  | 13500  | 6.1698          |
  | 6.2461        | 5.42  | 14000  | 6.1665          |
  | 6.2304        | 5.62  | 14500  | 6.1873          |
  | 6.2235        | 5.81  | 15000  | 6.1555          |
  | 6.2262        | 6.0   | 15500  | 6.1128          |
  | 6.2238        | 6.2   | 16000  | 6.1545          |
  | 6.2127        | 6.39  | 16500  | 6.1131          |
  | 6.221         | 6.58  | 17000  | 6.1513          |
  | 6.1974        | 6.78  | 17500  | 6.1712          |
  | 6.175         | 6.97  | 18000  | 6.1073          |
  | 6.2042        | 7.16  | 18500  | 6.1176          |
  | 6.1898        | 7.36  | 19000  | 6.0470          |
  | 6.1961        | 7.55  | 19500  | 6.1011          |
  | 6.1883        | 7.75  | 20000  | 6.1064          |
  | 6.2171        | 7.94  | 20500  | 6.1299          |
  | 6.175         | 8.13  | 21000  | 6.1313          |
  | 6.1757        | 8.33  | 21500  | 6.0899          |
  | 6.1776        | 8.52  | 22000  | 6.1196          |
  | 6.1377        | 8.71  | 22500  | 6.1554          |
  | 6.1688        | 8.91  | 23000  | 6.1037          |
  | 6.1555        | 9.1   | 23500  | 6.1622          |
  | 6.1665        | 9.3   | 24000  | 6.0622          |
  | 6.144         | 9.49  | 24500  | 6.0763          |
  | 6.1394        | 9.68  | 25000  | 6.0803          |
  | 6.1731        | 9.88  | 25500  | 6.1243          |
  | 6.1655        | 10.07 | 26000  | 6.0929          |
  | 6.1028        | 10.26 | 26500  | 6.1178          |
  | 6.1145        | 10.46 | 27000  | 6.1426          |
  | 6.1153        | 10.65 | 27500  | 6.1156          |
  | 6.1274        | 10.84 | 28000  | 6.0922          |
  | 6.1441        | 11.04 | 28500  | 6.0556          |
  | 6.1179        | 11.23 | 29000  | 6.1316          |
  | 6.1379        | 11.43 | 29500  | 6.0560          |
  | 6.1273        | 11.62 | 30000  | 6.1321          |
  | 6.1104        | 11.81 | 30500  | 6.1229          |
  | 6.1156        | 12.01 | 31000  | 6.0803          |
  | 6.0711        | 12.2  | 31500  | 6.0110          |
  | 6.1132        | 12.39 | 32000  | 6.1489          |
  | 6.065         | 12.59 | 32500  | 6.1082          |
  | 6.0774        | 12.78 | 33000  | 6.0590          |
  | 6.096         | 12.97 | 33500  | 6.0611          |
  | 6.1172        | 13.17 | 34000  | 6.0857          |
  | 6.0845        | 13.36 | 34500  | 6.0799          |
  | 6.0551        | 13.56 | 35000  | 6.0768          |
  | 6.0593        | 13.75 | 35500  | 6.0880          |
  | 6.0605        | 13.94 | 36000  | 6.0715          |
  | 6.0849        | 14.14 | 36500  | 5.9769          |
  | 6.0739        |