metadata

library_name: transformers
license: mit
base_model: FacebookAI/xlm-roberta-large
tags:
  - generated_from_trainer
metrics:
  - precision
  - recall
  - f1
  - accuracy
model-index:
  - name: xml-roberta-large-finetuned-corregido-tokenizadorES-mama
    results: []

xml-roberta-large-finetuned-corregido-tokenizadorES-mama

This model is a fine-tuned version of FacebookAI/xlm-roberta-large. It achieves the following results on the evaluation set:

Loss: 0.0460
Precision: 0.9110
Recall: 0.9394
F1: 0.9250
Accuracy: 0.9866

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 1e-05
train_batch_size: 8
eval_batch_size: 8
seed: 42
gradient_accumulation_steps: 8
total_train_batch_size: 64
optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: cosine_with_restarts
lr_scheduler_warmup_ratio: 0.2
num_epochs: 30
mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1	Accuracy
2.6643	1.0	86	2.5267	0.0	0.0	0.0	0.5837
1.9359	2.0	172	1.9628	0.0	0.0	0.0	0.6009
1.6777	3.0	258	1.6615	0.0357	0.0242	0.0289	0.6589
1.4374	4.0	344	1.3392	0.0945	0.0474	0.0632	0.7071
1.1194	5.0	430	1.0558	0.1712	0.1270	0.1458	0.7528
0.9551	6.0	516	0.8403	0.2826	0.2219	0.2486	0.7989
0.7683	7.0	602	0.6553	0.3573	0.3927	0.3742	0.8331
0.6506	8.0	688	0.4905	0.5	0.5087	0.5043	0.8727
0.5431	9.0	774	0.3954	0.5765	0.5941	0.5852	0.8949
0.4028	10.0	860	0.3061	0.6303	0.6632	0.6463	0.9178
0.3332	11.0	946	0.2540	0.6569	0.7296	0.6913	0.9313
0.2715	12.0	1032	0.2007	0.7223	0.7707	0.7457	0.9461
0.2678	13.0	1118	0.1619	0.7506	0.8013	0.7751	0.9557
0.2267	14.0	1204	0.1468	0.7608	0.8318	0.7948	0.9603
0.1875	15.0	1290	0.1357	0.7759	0.8413	0.8073	0.9640
0.1753	16.0	1376	0.1166	0.8112	0.8651	0.8372	0.9692
0.1616	17.0	1462	0.0967	0.8204	0.8788	0.8486	0.9731
0.1337	18.0	1548	0.0854	0.8389	0.8951	0.8661	0.9762
0.1298	19.0	1634	0.0676	0.8623	0.9014	0.8814	0.9804
0.1115	20.0	1720	0.0701	0.8687	0.9135	0.8905	0.9808
0.1139	21.0	1806	0.0602	0.8916	0.9278	0.9093	0.9830
0.114	22.0	1892	0.0543	0.8957	0.9278	0.9114	0.9842
0.0944	23.0	1978	0.0569	0.8922	0.9341	0.9127	0.9843
0.0893	24.0	2064	0.0517	0.8986	0.9346	0.9163	0.9852
0.0836	25.0	2150	0.0476	0.9057	0.9367	0.9210	0.9862
0.0841	26.0	2236	0.0489	0.9062	0.9367	0.9212	0.9859
0.0865	27.0	2322	0.0459	0.9095	0.9378	0.9234	0.9866
0.0859	28.0	2408	0.0464	0.9096	0.9394	0.9243	0.9866
0.0796	29.0	2494	0.0461	0.9101	0.9394	0.9245	0.9866
0.0774	29.6550	2550	0.0460	0.9110	0.9394	0.9250	0.9866

Framework versions

Transformers 4.50.3
Pytorch 2.6.0+cu124
Datasets 3.5.0
Tokenizers 0.21.1