readme: ---
  license: mit
  tags:
  - generated_from_trainer
  model-index:
  - name: BERiT_2000_custom_architecture_150_epochs_2
    results: []
  ---
  
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
  should probably proofread and complete it, then remove this comment. -->
  
  # BERiT_2000_custom_architecture_150_epochs_2
  
  This model is a fine-tuned version of [roberta-base](https://huggingface.co/roberta-base) on an unknown dataset.
  It achieves the following results on the evaluation set:
  - Loss: 4.0232
  
  ## Model description
  
  More information needed
  
  ## Intended uses & limitations
  
  More information needed
  
  ## Training and evaluation data
  
  More information needed
  
  ## Training procedure
  
  ### Training hyperparameters
  
  The following hyperparameters were used during training:
  - learning_rate: 0.0005
  - train_batch_size: 8
  - eval_batch_size: 8
  - seed: 42
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  - lr_scheduler_type: linear
  - num_epochs: 150
  
  ### Training results
  
  | Training Loss | Epoch  | Step   | Validation Loss |
  |:-------------:|:------:|:------:|:---------------:|
  | 16.9252       | 0.19   | 500    | 8.8086          |
  | 8.1417        | 0.39   | 1000   | 7.4588          |
  | 7.3432        | 0.58   | 1500   | 7.0258          |
  | 6.97          | 0.77   | 2000   | 6.8881          |
  | 6.8537        | 0.97   | 2500   | 6.7435          |
  | 6.7586        | 1.16   | 3000   | 6.5586          |
  | 6.6631        | 1.36   | 3500   | 6.4888          |
  | 6.5805        | 1.55   | 4000   | 6.3412          |
  | 6.5217        | 1.74   | 4500   | 6.3315          |
  | 6.4594        | 1.94   | 5000   | 6.2520          |
  | 6.4763        | 2.13   | 5500   | 6.3291          |
  | 6.4548        | 2.32   | 6000   | 6.2185          |
  | 6.3771        | 2.52   | 6500   | 6.2541          |
  | 6.404         | 2.71   | 7000   | 6.1994          |
  | 6.355         | 2.9    | 7500   | 6.2198          |
  | 6.3303        | 3.1    | 8000   | 6.2142          |
  | 6.3365        | 3.29   | 8500   | 6.2052          |
  | 6.3083        | 3.49   | 9000   | 6.1894          |
  | 6.318         | 3.68   | 9500   | 6.1603          |
  | 6.3098        | 3.87   | 10000  | 6.1916          |
  | 6.3333        | 4.07   | 10500  | 6.1624          |
  | 6.2708        | 4.26   | 11000  | 6.1487          |
  | 6.2791        | 4.45   | 11500  | 6.2454          |
  | 6.2689        | 4.65   | 12000  | 6.1647          |
  | 6.2697        | 4.84   | 12500  | 6.2024          |
  | 6.2483        | 5.03   | 13000  | 6.1729          |
  | 6.2427        | 5.23   | 13500  | 6.1592          |
  | 6.253         | 5.42   | 14000  | 6.1581          |
  | 6.2397        | 5.62   | 14500  | 6.1967          |
  | 6.2409        | 5.81   | 15000  | 6.1731          |
  | 6.2459        | 6.0    | 15500  | 6.1489          |
  | 6.2361        | 6.2    | 16000  | 6.1608          |
  | 6.2152        | 6.39   | 16500  | 6.1267          |
  | 6.229         | 6.58   | 17000  | 6.1480          |
  | 6.209         | 6.78   | 17500  | 6.1662          |
  | 6.18          | 6.97   | 18000  | 6.0836          |
  | 6.2165        | 7.16   | 18500  | 6.1457          |
  | 6.1968        | 7.36   | 19000  | 6.0355          |
  | 6.2039        | 7.55   | 19500  | 6.1197          |
  | 6.2061        | 7.75   | 20000  | 6.1013          |
  | 6.2161        | 7.94   | 20500  | 6.1334          |
  | 6.1798        | 8.13   | 21000  | 6.1302          |
  | 6.1736        | 8.33   | 21500  | 6.1075          |
  | 6.1815        | 8.52   | 22000  | 6.1220          |
  | 6.1441        | 8.71   | 22500  | 6.1518          |
  | 6.1752        | 8.91   | 23000  | 6.1060          |
  | 6.1577        | 9.1    | 23500  | 6.1746          |
  | 6.1739        | 9.3    | 24000  | 6.0522          |
  | 6.1406        | 9.49   | 24500  | 6.0760          |
  | 6.1363        | 9.68   | 25000  | 6.0854          |
  | 6.1694        | 9.88   | 25500  | 6.1197          |
  | 6.1573        | 10.07  | 26000  | 6.1097          |
  | 6.0988        | 10.26  | 26500  | 6.1068          |
  | 6.1241        | 10.46  | 27000  | 6.1477          |
  | 6.1162        | 10.65  | 27500  | 6.1261          |
  | 6.1142        | 10.84  | 28000  | 6.1454          |
  | 6.1471        | 11.04  | 28500  | 6.0853          |
  | 6.1119        | 11.23  | 29000  | 6.1412          |
  | 6.1424        | 11.43  | 29500  | 6.0572          |
  | 6.1304        | 11.62  | 30000  | 6.1497          |
  | 6.1205        | 11.81  | 30500  | 6.1240          |
  | 6.1217        | 12.01  | 31000  | 6.0854          |
  | 6.0652        | 12.2   | 31500  | 6.0118          |
  | 6.1076        | 12.39  | 32000  | 6.1618          |
  | 6.0808        | 12.59  | 32500  | 6.1147          |
  | 6.0869        | 12.78  | 33000  | 6.0711          |
  | 6.1058        | 12.97  | 33500  | 6.0823          |
  | 6.1286        | 13.17  | 34000  | 6.0848          |
  | 6.0982        | 13.36  | 34500  | 6.1015          |
  | 6.0668        | 13.56  | 35000  | 6.0999          |
  | 6.0736        | 13.75  | 35500  | 6.1143          |
  | 6.0617        | 13.94  | 