readme: ---
  license: mit
  tags:
  - generated_from_trainer
  model-index:
  - name: BERiT_2000_custom_architecture_100_epochs
    results: []
  ---
  
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
  should probably proofread and complete it, then remove this comment. -->
  
  # BERiT_2000_custom_architecture_100_epochs
  
  This model is a fine-tuned version of [roberta-base](https://huggingface.co/roberta-base) on an unknown dataset.
  It achieves the following results on the evaluation set:
  - Loss: 4.3615
  
  ## Model description
  
  More information needed
  
  ## Intended uses & limitations
  
  More information needed
  
  ## Training and evaluation data
  
  More information needed
  
  ## Training procedure
  
  ### Training hyperparameters
  
  The following hyperparameters were used during training:
  - learning_rate: 0.0005
  - train_batch_size: 8
  - eval_batch_size: 8
  - seed: 42
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  - lr_scheduler_type: linear
  - num_epochs: 100
  
  ### Training results
  
  | Training Loss | Epoch | Step   | Validation Loss |
  |:-------------:|:-----:|:------:|:---------------:|
  | 16.6444       | 0.19  | 500    | 8.8209          |
  | 8.1904        | 0.39  | 1000   | 7.5197          |
  | 7.3572        | 0.58  | 1500   | 7.1037          |
  | 7.0042        | 0.77  | 2000   | 6.8817          |
  | 6.8626        | 0.97  | 2500   | 6.7353          |
  | 6.7582        | 1.16  | 3000   | 6.5802          |
  | 6.6345        | 1.36  | 3500   | 6.4232          |
  | 6.5817        | 1.55  | 4000   | 6.3375          |
  | 6.517         | 1.74  | 4500   | 6.3439          |
  | 6.4531        | 1.94  | 5000   | 6.2870          |
  | 6.469         | 2.13  | 5500   | 6.3208          |
  | 6.4503        | 2.32  | 6000   | 6.2136          |
  | 6.3679        | 2.52  | 6500   | 6.2260          |
  | 6.4032        | 2.71  | 7000   | 6.2015          |
  | 6.357         | 2.9   | 7500   | 6.2363          |
  | 6.3349        | 3.1   | 8000   | 6.2101          |
  | 6.342         | 3.29  | 8500   | 6.2031          |
  | 6.3047        | 3.49  | 9000   | 6.1945          |
  | 6.3204        | 3.68  | 9500   | 6.1681          |
  | 6.2935        | 3.87  | 10000  | 6.1999          |
  | 6.3319        | 4.07  | 10500  | 6.1613          |
  | 6.2528        | 4.26  | 11000  | 6.1354          |
  | 6.2683        | 4.45  | 11500  | 6.2427          |
  | 6.2572        | 4.65  | 12000  | 6.1477          |
  | 6.2509        | 4.84  | 12500  | 6.1770          |
  | 6.2402        | 5.03  | 13000  | 6.1779          |
  | 6.2412        | 5.23  | 13500  | 6.1516          |
  | 6.2291        | 5.42  | 14000  | 6.1498          |
  | 6.2203        | 5.62  | 14500  | 6.1804          |
  | 6.2341        | 5.81  | 15000  | 6.1501          |
  | 6.2242        | 6.0   | 15500  | 6.1239          |
  | 6.2163        | 6.2   | 16000  | 6.1567          |
  | 6.2079        | 6.39  | 16500  | 6.1188          |
  | 6.2176        | 6.58  | 17000  | 6.1620          |
  | 6.1926        | 6.78  | 17500  | 6.1635          |
  | 6.1743        | 6.97  | 18000  | 6.0749          |
  | 6.1978        | 7.16  | 18500  | 6.1316          |
  | 6.1868        | 7.36  | 19000  | 6.0297          |
  | 6.19          | 7.55  | 19500  | 6.1126          |
  | 6.2005        | 7.75  | 20000  | 6.0985          |
  | 6.2056        | 7.94  | 20500  | 6.1100          |
  | 6.1628        | 8.13  | 21000  | 6.1321          |
  | 6.169         | 8.33  | 21500  | 6.0842          |
  | 6.1636        | 8.52  | 22000  | 6.1205          |
  | 6.1278        | 8.71  | 22500  | 6.1270          |
  | 6.1656        | 8.91  | 23000  | 6.1049          |
  | 6.1526        | 9.1   | 23500  | 6.1462          |
  | 6.1624        | 9.3   | 24000  | 6.0534          |
  | 6.1353        | 9.49  | 24500  | 6.0862          |
  | 6.1264        | 9.68  | 25000  | 6.0844          |
  | 6.1648        | 9.88  | 25500  | 6.1206          |
  | 6.1574        | 10.07 | 26000  | 6.0942          |
  | 6.0971        | 10.26 | 26500  | 6.1151          |
  | 6.119         | 10.46 | 27000  | 6.1148          |
  | 6.1217        | 10.65 | 27500  | 6.1076          |
  | 6.1054        | 10.84 | 28000  | 6.1457          |
  | 6.1402        | 11.04 | 28500  | 6.0442          |
  | 6.1124        | 11.23 | 29000  | 6.1404          |
  | 6.1457        | 11.43 | 29500  | 6.0622          |
  | 6.1248        | 11.62 | 30000  | 6.1377          |
  | 6.1204        | 11.81 | 30500  | 6.1056          |
  | 6.1097        | 12.01 | 31000  | 6.0780          |
  | 6.0713        | 12.2  | 31500  | 6.0061          |
  | 6.1119        | 12.39 | 32000  | 6.1671          |
  | 6.0744        | 12.59 | 32500  | 6.1235          |
  | 6.082         | 12.78 | 33000  | 6.0905          |
  | 6.0962        | 12.97 | 33500  | 6.0936          |
  | 6.1265        | 13.17 | 34000  | 6.0786          |
  | 6.0941        | 13.36 | 34500  | 6.0944          |
  | 6.0694        | 13.56 | 35000  | 6.0988          |
  | 6.0784        | 13.75 | 35500  | 6.1221          |
  | 6.0749        | 13.94 | 36000  | 6.0961          |
  | 6.103         | 14.14 | 36500  | 6.0141          