---
library_name: transformers
license: mit
base_model: roberta-base
tags:
- generated_from_trainer
metrics:
- accuracy
model-index:
- name: SST-2-FULL_FT-seed10
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# SST-2-FULL_FT-seed10

This model is a fine-tuned version of [roberta-base](https://huggingface.co/roberta-base) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 0.3049
- Accuracy: 0.9415

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 32
- eval_batch_size: 32
- seed: 42
- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- num_epochs: 5

### Training results

| Training Loss | Epoch  | Step  | Validation Loss | Accuracy |
|:-------------:|:------:|:-----:|:---------------:|:--------:|
| 0.4039        | 0.0950 | 200   | 0.2241          | 0.9140   |
| 0.2971        | 0.1900 | 400   | 0.2148          | 0.9163   |
| 0.2716        | 0.2850 | 600   | 0.2114          | 0.9266   |
| 0.2327        | 0.3800 | 800   | 0.2143          | 0.9232   |
| 0.2222        | 0.4751 | 1000  | 0.3102          | 0.9048   |
| 0.2166        | 0.5701 | 1200  | 0.2533          | 0.9220   |
| 0.2112        | 0.6651 | 1400  | 0.2210          | 0.9312   |
| 0.199         | 0.7601 | 1600  | 0.2412          | 0.9255   |
| 0.205         | 0.8551 | 1800  | 0.2204          | 0.9289   |
| 0.188         | 0.9501 | 2000  | 0.2112          | 0.9346   |
| 0.1783        | 1.0451 | 2200  | 0.2042          | 0.9369   |
| 0.1502        | 1.1401 | 2400  | 0.1916          | 0.9392   |
| 0.155         | 1.2352 | 2600  | 0.2369          | 0.9335   |
| 0.1417        | 1.3302 | 2800  | 0.2446          | 0.9300   |
| 0.1452        | 1.4252 | 3000  | 0.2524          | 0.9255   |
| 0.1454        | 1.5202 | 3200  | 0.2622          | 0.9255   |
| 0.1517        | 1.6152 | 3400  | 0.1906          | 0.9255   |
| 0.1396        | 1.7102 | 3600  | 0.2471          | 0.9209   |
| 0.1422        | 1.8052 | 3800  | 0.2245          | 0.9323   |
| 0.1522        | 1.9002 | 4000  | 0.2231          | 0.9358   |
| 0.1438        | 1.9952 | 4200  | 0.2113          | 0.9381   |
| 0.1173        | 2.0903 | 4400  | 0.2444          | 0.9346   |
| 0.1116        | 2.1853 | 4600  | 0.2508          | 0.9335   |
| 0.1085        | 2.2803 | 4800  | 0.3127          | 0.9266   |
| 0.1123        | 2.3753 | 5000  | 0.2474          | 0.9243   |
| 0.1067        | 2.4703 | 5200  | 0.2621          | 0.9300   |
| 0.1018        | 2.5653 | 5400  | 0.2460          | 0.9358   |
| 0.1232        | 2.6603 | 5600  | 0.2445          | 0.9358   |
| 0.097         | 2.7553 | 5800  | 0.2641          | 0.9289   |
| 0.1085        | 2.8504 | 6000  | 0.2546          | 0.9300   |
| 0.0959        | 2.9454 | 6200  | 0.2663          | 0.9358   |
| 0.0961        | 3.0404 | 6400  | 0.2741          | 0.9358   |
| 0.0818        | 3.1354 | 6600  | 0.2428          | 0.9369   |
| 0.0791        | 3.2304 | 6800  | 0.2717          | 0.9346   |
| 0.0838        | 3.3254 | 7000  | 0.2503          | 0.9358   |
| 0.0815        | 3.4204 | 7200  | 0.2384          | 0.9392   |
| 0.0763        | 3.5154 | 7400  | 0.3029          | 0.9312   |
| 0.0776        | 3.6105 | 7600  | 0.2866          | 0.9335   |
| 0.0853        | 3.7055 | 7800  | 0.2533          | 0.9369   |
| 0.088         | 3.8005 | 8000  | 0.2809          | 0.9335   |
| 0.0775        | 3.8955 | 8200  | 0.2812          | 0.9369   |
| 0.081         | 3.9905 | 8400  | 0.2522          | 0.9346   |
| 0.0581        | 4.0855 | 8600  | 0.2981          | 0.9323   |
| 0.0484        | 4.1805 | 8800  | 0.3132          | 0.9369   |
| 0.0649        | 4.2755 | 9000  | 0.3049          | 0.9415   |
| 0.0644        | 4.3705 | 9200  | 0.2986          | 0.9358   |
| 0.0607        | 4.4656 | 9400  | 0.2718          | 0.9404   |
| 0.0533        | 4.5606 | 9600  | 0.2927          | 0.9392   |
| 0.0641        | 4.6556 | 9800  | 0.2875          | 0.9381   |
| 0.0605        | 4.7506 | 10000 | 0.2811          | 0.9381   |
| 0.0611        | 4.8456 | 10200 | 0.2845          | 0.9358   |
| 0.0575        | 4.9406 | 10400 | 0.2866          | 0.9381   |


### Framework versions

- Transformers 4.54.1
- Pytorch 2.5.1+cu121
- Datasets 4.0.0
- Tokenizers 0.21.4