roberta-large-sst-2-32-13-30

This model is a fine-tuned version of roberta-large on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.8494
• Accuracy: 0.6406

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1.5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 5
• num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	1.0	2	0.7123	0.5
No log	2.0	4	0.7030	0.5
No log	3.0	6	0.6935	0.5
No log	4.0	8	0.6906	0.5312
0.718	5.0	10	0.6893	0.6094
0.718	6.0	12	0.6883	0.5625
0.718	7.0	14	0.6860	0.5469
0.718	8.0	16	0.6811	0.6094
0.718	9.0	18	0.6780	0.5781
0.6565	10.0	20	0.6859	0.5469
0.6565	11.0	22	0.6943	0.5469
0.6565	12.0	24	0.7061	0.5469
0.6565	13.0	26	0.6963	0.5469
0.6565	14.0	28	0.7058	0.5781
0.5726	15.0	30	0.7036	0.5938
0.5726	16.0	32	0.7185	0.6094
0.5726	17.0	34	0.7307	0.6094
0.5726	18.0	36	0.7743	0.6094
0.5726	19.0	38	0.7790	0.5938
0.4219	20.0	40	0.7805	0.6094
0.4219	21.0	42	0.7744	0.6094
0.4219	22.0	44	0.7960	0.5938
0.4219	23.0	46	0.8495	0.6094
0.4219	24.0	48	0.8893	0.5938
0.3261	25.0	50	0.8901	0.625
0.3261	26.0	52	0.8924	0.625
0.3261	27.0	54	0.8908	0.6094
0.3261	28.0	56	0.8769	0.6094
0.3261	29.0	58	0.8592	0.6094
0.2415	30.0	60	0.8494	0.6406

Framework versions

• Transformers 4.32.0.dev0
• Pytorch 2.0.1+cu118
• Datasets 2.4.0
• Tokenizers 0.13.3