readme: ---
  license: mit
  tags:
  - generated_from_trainer
  model-index:
  - name: BERiT_2000_custom_architecture
    results: []
  ---
  
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
  should probably proofread and complete it, then remove this comment. -->
  
  # BERiT_2000_custom_architecture
  
  This model is a fine-tuned version of [roberta-base](https://huggingface.co/roberta-base) on an unknown dataset.
  It achieves the following results on the evaluation set:
  - Loss: 6.0153
  
  ## Model description
  
  More information needed
  
  ## Intended uses & limitations
  
  More information needed
  
  ## Training and evaluation data
  
  More information needed
  
  ## Training procedure
  
  ### Training hyperparameters
  
  The following hyperparameters were used during training:
  - learning_rate: 0.0005
  - train_batch_size: 8
  - eval_batch_size: 8
  - seed: 42
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  - lr_scheduler_type: linear
  - num_epochs: 10
  
  ### Training results
  
  | Training Loss | Epoch | Step  | Validation Loss |
  |:-------------:|:-----:|:-----:|:---------------:|
  | 16.6991       | 0.19  | 500   | 8.9825          |
  | 8.259         | 0.39  | 1000  | 7.5650          |
  | 7.3895        | 0.58  | 1500  | 7.1084          |
  | 7.0328        | 0.77  | 2000  | 6.8799          |
  | 6.8743        | 0.97  | 2500  | 6.7598          |
  | 6.7775        | 1.16  | 3000  | 6.5915          |
  | 6.6348        | 1.36  | 3500  | 6.4513          |
  | 6.5759        | 1.55  | 4000  | 6.3394          |
  | 6.5243        | 1.74  | 4500  | 6.3336          |
  | 6.4492        | 1.94  | 5000  | 6.2714          |
  | 6.4472        | 2.13  | 5500  | 6.2921          |
  | 6.4283        | 2.32  | 6000  | 6.1922          |
  | 6.3508        | 2.52  | 6500  | 6.2112          |
  | 6.3838        | 2.71  | 7000  | 6.1727          |
  | 6.3303        | 2.9   | 7500  | 6.2093          |
  | 6.3067        | 3.1   | 8000  | 6.1984          |
  | 6.3099        | 3.29  | 8500  | 6.1589          |
  | 6.2806        | 3.49  | 9000  | 6.1732          |
  | 6.2861        | 3.68  | 9500  | 6.1257          |
  | 6.2645        | 3.87  | 10000 | 6.1655          |
  | 6.2992        | 4.07  | 10500 | 6.1156          |
  | 6.2331        | 4.26  | 11000 | 6.1212          |
  | 6.2247        | 4.45  | 11500 | 6.1991          |
  | 6.2235        | 4.65  | 12000 | 6.1181          |
  | 6.2354        | 4.84  | 12500 | 6.1469          |
  | 6.2157        | 5.03  | 13000 | 6.1170          |
  | 6.2076        | 5.23  | 13500 | 6.1128          |
  | 6.2085        | 5.42  | 14000 | 6.1079          |
  | 6.1917        | 5.62  | 14500 | 6.1511          |
  | 6.1917        | 5.81  | 15000 | 6.1032          |
  | 6.1887        | 6.0   | 15500 | 6.0877          |
  | 6.1895        | 6.2   | 16000 | 6.0876          |
  | 6.1685        | 6.39  | 16500 | 6.0734          |
  | 6.1709        | 6.58  | 17000 | 6.1039          |
  | 6.1442        | 6.78  | 17500 | 6.1347          |
  | 6.126         | 6.97  | 18000 | 6.0571          |
  | 6.1587        | 7.16  | 18500 | 6.0808          |
  | 6.1349        | 7.36  | 19000 | 5.9921          |
  | 6.1487        | 7.55  | 19500 | 6.0548          |
  | 6.1362        | 7.75  | 20000 | 6.0746          |
  | 6.1581        | 7.94  | 20500 | 6.0689          |
  | 6.1225        | 8.13  | 21000 | 6.0916          |
  | 6.1233        | 8.33  | 21500 | 6.0504          |
  | 6.1192        | 8.52  | 22000 | 6.0630          |
  | 6.0843        | 8.71  | 22500 | 6.0927          |
  | 6.1144        | 8.91  | 23000 | 6.0464          |
  | 6.1012        | 9.1   | 23500 | 6.0872          |
  | 6.1118        | 9.3   | 24000 | 6.0153          |
  
  
  ### Framework versions
  
  - Transformers 4.24.0
  - Pytorch 1.12.1+cu113
  - Datasets 2.6.1
  - Tokenizers 0.13.2
  