readme: ---
  license: mit
  tags:
  - generated_from_trainer
  model-index:
  - name: BERiT_2000_custom_architecture_300_epochs_ls_.1
    results: []
  ---
  
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
  should probably proofread and complete it, then remove this comment. -->
  
  # BERiT_2000_custom_architecture_300_epochs_ls_.1
  
  This model is a fine-tuned version of [roberta-base](https://huggingface.co/roberta-base) on an unknown dataset.
  It achieves the following results on the evaluation set:
  - Loss: 4.5772
  
  ## Model description
  
  More information needed
  
  ## Intended uses & limitations
  
  More information needed
  
  ## Training and evaluation data
  
  More information needed
  
  ## Training procedure
  
  ### Training hyperparameters
  
  The following hyperparameters were used during training:
  - learning_rate: 0.0005
  - train_batch_size: 8
  - eval_batch_size: 8
  - seed: 42
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  - lr_scheduler_type: linear
  - num_epochs: 300
  - label_smoothing_factor: 0.1
  
  ### Training results
  
  | Training Loss | Epoch  | Step   | Validation Loss |
  |:-------------:|:------:|:------:|:---------------:|
  | 16.0386       | 0.19   | 500    | 8.5964          |
  | 7.9494        | 0.39   | 1000   | 7.3307          |
  | 7.2772        | 0.58   | 1500   | 7.0026          |
  | 7.0334        | 0.77   | 2000   | 6.9589          |
  | 6.9592        | 0.97   | 2500   | 6.8756          |
  | 6.8939        | 1.16   | 3000   | 6.7537          |
  | 6.8257        | 1.36   | 3500   | 6.6525          |
  | 6.77          | 1.55   | 4000   | 6.5600          |
  | 6.7312        | 1.74   | 4500   | 6.5832          |
  | 6.6799        | 1.94   | 5000   | 6.5013          |
  | 6.6882        | 2.13   | 5500   | 6.5632          |
  | 6.6718        | 2.32   | 6000   | 6.4732          |
  | 6.5983        | 2.52   | 6500   | 6.4838          |
  | 6.6341        | 2.71   | 7000   | 6.4504          |
  | 6.5868        | 2.9    | 7500   | 6.4702          |
  | 6.5685        | 3.1    | 8000   | 6.4778          |
  | 6.571         | 3.29   | 8500   | 6.4604          |
  | 6.5488        | 3.49   | 9000   | 6.4622          |
  | 6.5659        | 3.68   | 9500   | 6.4282          |
  | 6.541         | 3.87   | 10000  | 6.4521          |
  | 6.5668        | 4.07   | 10500  | 6.4167          |
  | 6.5281        | 4.26   | 11000  | 6.3849          |
  | 6.5203        | 4.45   | 11500  | 6.4916          |
  | 6.5046        | 4.65   | 12000  | 6.4301          |
  | 6.5164        | 4.84   | 12500  | 6.4422          |
  | 6.5047        | 5.03   | 13000  | 6.4398          |
  | 6.5047        | 5.23   | 13500  | 6.4256          |
  | 6.4966        | 5.42   | 14000  | 6.4107          |
  | 6.4923        | 5.62   | 14500  | 6.4669          |
  | 6.488         | 5.81   | 15000  | 6.4339          |
  | 6.4912        | 6.0    | 15500  | 6.4071          |
  | 6.4929        | 6.2    | 16000  | 6.4216          |
  | 6.4831        | 6.39   | 16500  | 6.4074          |
  | 6.49          | 6.58   | 17000  | 6.4300          |
  | 6.468         | 6.78   | 17500  | 6.4405          |
  | 6.4559        | 6.97   | 18000  | 6.3703          |
  | 6.4797        | 7.16   | 18500  | 6.4108          |
  | 6.4658        | 7.36   | 19000  | 6.3464          |
  | 6.4665        | 7.55   | 19500  | 6.3833          |
  | 6.4686        | 7.75   | 20000  | 6.4091          |
  | 6.4802        | 7.94   | 20500  | 6.4008          |
  | 6.4514        | 8.13   | 21000  | 6.4347          |
  | 6.4439        | 8.33   | 21500  | 6.3952          |
  | 6.4502        | 8.52   | 22000  | 6.4035          |
  | 6.4222        | 8.71   | 22500  | 6.4276          |
  | 6.4497        | 8.91   | 23000  | 6.4099          |
  | 6.4319        | 9.1    | 23500  | 6.4499          |
  | 6.4404        | 9.3    | 24000  | 6.3507          |
  | 6.4153        | 9.49   | 24500  | 6.3884          |
  | 6.4103        | 9.68   | 25000  | 6.3713          |
  | 6.4381        | 9.88   | 25500  | 6.4313          |
  | 6.4321        | 10.07  | 26000  | 6.3937          |
  | 6.3856        | 10.26  | 26500  | 6.4040          |
  | 6.3995        | 10.46  | 27000  | 6.4348          |
  | 6.3904        | 10.65  | 27500  | 6.4194          |
  | 6.3988        | 10.84  | 28000  | 6.4226          |
  | 6.4134        | 11.04  | 28500  | 6.3449          |
  | 6.402         | 11.23  | 29000  | 6.4220          |
  | 6.4217        | 11.43  | 29500  | 6.3570          |
  | 6.398         | 11.62  | 30000  | 6.4093          |
  | 6.3865        | 11.81  | 30500  | 6.3905          |
  | 6.3913        | 12.01  | 31000  | 6.3530          |
  | 6.3449        | 12.2   | 31500  | 6.3180          |
  | 6.3886        | 12.39  | 32000  | 6.4239          |
  | 6.3437        | 12.59  | 32500  | 6.4005          |
  | 6.3592        | 12.78  | 33000  | 6.3495          |
  | 6.3715        | 12.97  | 33500  | 6.3748          |
  | 6.3839        | 13.17  | 34000  | 6.3552          |
  | 6.3611        | 13.36  | 34500  | 6.3749          |
  | 6.331         | 13.56  | 35000  | 6.3625          |
  | 6.3389        | 13.75  | 35500  | 6.3887 