test11_balanced_50ep

This model is a fine-tuned version of ibaucells/RoBERTa-ca-CaWikiTC on the None dataset. It achieves the following results on the evaluation set:

• Loss: 2.0764
• Accuracy: 0.5294
• F1: 0.5256

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 10
• eval_batch_size: 10
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
2.8494	1.0	70	2.8328	0.0588	0.0065
2.8418	2.0	140	2.8317	0.0588	0.0067
2.8254	3.0	210	2.8274	0.0647	0.0110
2.8363	4.0	280	2.8299	0.0529	0.0233
2.8395	5.0	350	2.8267	0.0765	0.0465
2.8157	6.0	420	2.8273	0.0824	0.0654
2.7264	7.0	490	2.7849	0.1059	0.0680
2.6251	8.0	560	2.7422	0.1529	0.0842
2.5134	9.0	630	2.6863	0.1412	0.0848
2.4186	10.0	700	2.6497	0.1765	0.1244
2.3482	11.0	770	2.6209	0.1471	0.0887
2.1749	12.0	840	2.6347	0.1529	0.1056
2.1659	13.0	910	2.5642	0.2176	0.1774
1.9836	14.0	980	2.5690	0.2294	0.1915
1.9313	15.0	1050	2.5759	0.2588	0.2225
1.7556	16.0	1120	2.5060	0.2529	0.2016
1.6241	17.0	1190	2.3977	0.3353	0.3119
1.5035	18.0	1260	2.4009	0.3059	0.2652
1.4646	19.0	1330	2.2717	0.3824	0.3651
1.3803	20.0	1400	2.3232	0.3529	0.3405
1.2347	21.0	1470	2.2476	0.3941	0.3904
1.1416	22.0	1540	2.2397	0.3706	0.3551
1.0879	23.0	1610	2.2348	0.3588	0.3546
0.9444	24.0	1680	2.1676	0.4118	0.4154
0.8462	25.0	1750	2.0285	0.4824	0.4928
0.7912	26.0	1820	2.0156	0.4765	0.4724
0.7106	27.0	1890	2.0294	0.4882	0.4871
0.6734	28.0	1960	2.0707	0.4588	0.4588
0.571	29.0	2030	1.9563	0.5059	0.5114
0.5605	30.0	2100	1.9666	0.5176	0.5124
0.474	31.0	2170	1.9746	0.5118	0.5187
0.4498	32.0	2240	2.0250	0.5059	0.5016
0.3727	33.0	2310	1.9343	0.5294	0.5366
0.4182	34.0	2380	1.9686	0.5176	0.5163
0.3712	35.0	2450	1.9111	0.5294	0.5329
0.3055	36.0	2520	1.8849	0.5353	0.5365
0.2612	37.0	2590	2.0725	0.5235	0.5175
0.2258	38.0	2660	1.9393	0.5294	0.5290
0.175	39.0	2730	1.9869	0.5294	0.5265
0.1632	40.0	2800	2.0229	0.5118	0.5090
0.2105	41.0	2870	1.9864	0.5176	0.5151
0.173	42.0	2940	2.0138	0.5294	0.5278
0.1341	43.0	3010	2.0270	0.5529	0.5473
0.1599	44.0	3080	2.0372	0.5412	0.5376
0.1529	45.0	3150	2.0411	0.5412	0.5353
0.1248	46.0	3220	2.0776	0.5235	0.5213
0.1761	47.0	3290	2.0376	0.5294	0.5271
0.134	48.0	3360	2.0721	0.5294	0.5256
0.1014	49.0	3430	2.0854	0.5294	0.5247
0.1106	50.0	3500	2.0764	0.5294	0.5256

Framework versions

• Transformers 4.38.2
• Pytorch 2.1.0+cu121
• Datasets 2.18.0
• Tokenizers 0.15.2