readme: ---
  license: mit
  tags:
  - generated_from_trainer
  model-index:
  - name: BERiT_2000_custom_architecture_300_epochs
    results: []
  ---
  
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
  should probably proofread and complete it, then remove this comment. -->
  
  # BERiT_2000_custom_architecture_300_epochs
  
  This model is a fine-tuned version of [roberta-base](https://huggingface.co/roberta-base) on an unknown dataset.
  It achieves the following results on the evaluation set:
  - Loss: 3.7539
  
  ## Model description
  
  More information needed
  
  ## Intended uses & limitations
  
  More information needed
  
  ## Training and evaluation data
  
  More information needed
  
  ## Training procedure
  
  ### Training hyperparameters
  
  The following hyperparameters were used during training:
  - learning_rate: 0.0005
  - train_batch_size: 8
  - eval_batch_size: 8
  - seed: 42
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  - lr_scheduler_type: linear
  - num_epochs: 300
  
  ### Training results
  
  | Training Loss | Epoch  | Step   | Validation Loss |
  |:-------------:|:------:|:------:|:---------------:|
  | 16.9522       | 0.19   | 500    | 8.9110          |
  | 8.1663        | 0.39   | 1000   | 7.6129          |
  | 7.4169        | 0.58   | 1500   | 7.0763          |
  | 6.986         | 0.77   | 2000   | 6.8622          |
  | 6.867         | 0.97   | 2500   | 6.7466          |
  | 6.7472        | 1.16   | 3000   | 6.5698          |
  | 6.6284        | 1.36   | 3500   | 6.4532          |
  | 6.5708        | 1.55   | 4000   | 6.3080          |
  | 6.5123        | 1.74   | 4500   | 6.3716          |
  | 6.4619        | 1.94   | 5000   | 6.2711          |
  | 6.4596        | 2.13   | 5500   | 6.3147          |
  | 6.4407        | 2.32   | 6000   | 6.2327          |
  | 6.3668        | 2.52   | 6500   | 6.2272          |
  | 6.4037        | 2.71   | 7000   | 6.1710          |
  | 6.3446        | 2.9    | 7500   | 6.2018          |
  | 6.3164        | 3.1    | 8000   | 6.2128          |
  | 6.3208        | 3.29   | 8500   | 6.1826          |
  | 6.2957        | 3.49   | 9000   | 6.1787          |
  | 6.3112        | 3.68   | 9500   | 6.1344          |
  | 6.2746        | 3.87   | 10000  | 6.1777          |
  | 6.3207        | 4.07   | 10500  | 6.1327          |
  | 6.2534        | 4.26   | 11000  | 6.1097          |
  | 6.2566        | 4.45   | 11500  | 6.2141          |
  | 6.2536        | 4.65   | 12000  | 6.1309          |
  | 6.2531        | 4.84   | 12500  | 6.1652          |
  | 6.2389        | 5.03   | 13000  | 6.1563          |
  | 6.2337        | 5.23   | 13500  | 6.1723          |
  | 6.2347        | 5.42   | 14000  | 6.1454          |
  | 6.2244        | 5.62   | 14500  | 6.1824          |
  | 6.217         | 5.81   | 15000  | 6.1406          |
  | 6.2143        | 6.0    | 15500  | 6.1105          |
  | 6.2238        | 6.2    | 16000  | 6.1406          |
  | 6.2021        | 6.39   | 16500  | 6.0873          |
  | 6.209         | 6.58   | 17000  | 6.1186          |
  | 6.1927        | 6.78   | 17500  | 6.1397          |
  | 6.1669        | 6.97   | 18000  | 6.0674          |
  | 6.1938        | 7.16   | 18500  | 6.1061          |
  | 6.1861        | 7.36   | 19000  | 6.0005          |
  | 6.1896        | 7.55   | 19500  | 6.0958          |
  | 6.1903        | 7.75   | 20000  | 6.1051          |
  | 6.1975        | 7.94   | 20500  | 6.1247          |
  | 6.1668        | 8.13   | 21000  | 6.1151          |
  | 6.1692        | 8.33   | 21500  | 6.0780          |
  | 6.1641        | 8.52   | 22000  | 6.0889          |
  | 6.1378        | 8.71   | 22500  | 6.1192          |
  | 6.1697        | 8.91   | 23000  | 6.1112          |
  | 6.1462        | 9.1    | 23500  | 6.1386          |
  | 6.16          | 9.3    | 24000  | 6.0589          |
  | 6.1291        | 9.49   | 24500  | 6.0793          |
  | 6.1246        | 9.68   | 25000  | 6.0752          |
  | 6.1592        | 9.88   | 25500  | 6.1065          |
  | 6.1494        | 10.07  | 26000  | 6.0758          |
  | 6.0946        | 10.26  | 26500  | 6.1001          |
  | 6.1127        | 10.46  | 27000  | 6.1084          |
  | 6.0995        | 10.65  | 27500  | 6.1079          |
  | 6.1205        | 10.84  | 28000  | 6.0846          |
  | 6.1286        | 11.04  | 28500  | 6.0344          |
  | 6.1081        | 11.23  | 29000  | 6.1353          |
  | 6.1403        | 11.43  | 29500  | 6.0531          |
  | 6.1206        | 11.62  | 30000  | 6.1139          |
  | 6.1109        | 11.81  | 30500  | 6.0940          |
  | 6.1029        | 12.01  | 31000  | 6.0823          |
  | 6.0578        | 12.2   | 31500  | 6.0052          |
  | 6.096         | 12.39  | 32000  | 6.1617          |
  | 6.0679        | 12.59  | 32500  | 6.0964          |
  | 6.0785        | 12.78  | 33000  | 6.0737          |
  | 6.0874        | 12.97  | 33500  | 6.0553          |
  | 6.0981        | 13.17  | 34000  | 6.0471          |
  | 6.0767        | 13.36  | 34500  | 6.0850          |
  | 6.0511        | 13.56  | 35000  | 6.0626          |
  | 6.0567        | 13.75  | 35500  | 6.0790          |
  | 6.0494        | 13.94  | 3600