---
library_name: transformers
license: mit
base_model: FacebookAI/xlm-roberta-large
tags:
- generated_from_trainer
metrics:
- precision
- recall
- f1
- accuracy
model-index:
- name: xml-roberta-large-finetuned-corregido-tokenizadorES-mama
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# xml-roberta-large-finetuned-corregido-tokenizadorES-mama

This model is a fine-tuned version of [FacebookAI/xlm-roberta-large](https://huggingface.co/FacebookAI/xlm-roberta-large).
It achieves the following results on the evaluation set:
- Loss: 0.0460
- Precision: 0.9110
- Recall: 0.9394
- F1: 0.9250
- Accuracy: 0.9866


### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 1e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 8
- total_train_batch_size: 64
- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: cosine_with_restarts
- lr_scheduler_warmup_ratio: 0.2
- num_epochs: 30
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch   | Step | Validation Loss | Precision | Recall | F1     | Accuracy |
|:-------------:|:-------:|:----:|:---------------:|:---------:|:------:|:------:|:--------:|
| 2.6643        | 1.0     | 86   | 2.5267          | 0.0       | 0.0    | 0.0    | 0.5837   |
| 1.9359        | 2.0     | 172  | 1.9628          | 0.0       | 0.0    | 0.0    | 0.6009   |
| 1.6777        | 3.0     | 258  | 1.6615          | 0.0357    | 0.0242 | 0.0289 | 0.6589   |
| 1.4374        | 4.0     | 344  | 1.3392          | 0.0945    | 0.0474 | 0.0632 | 0.7071   |
| 1.1194        | 5.0     | 430  | 1.0558          | 0.1712    | 0.1270 | 0.1458 | 0.7528   |
| 0.9551        | 6.0     | 516  | 0.8403          | 0.2826    | 0.2219 | 0.2486 | 0.7989   |
| 0.7683        | 7.0     | 602  | 0.6553          | 0.3573    | 0.3927 | 0.3742 | 0.8331   |
| 0.6506        | 8.0     | 688  | 0.4905          | 0.5       | 0.5087 | 0.5043 | 0.8727   |
| 0.5431        | 9.0     | 774  | 0.3954          | 0.5765    | 0.5941 | 0.5852 | 0.8949   |
| 0.4028        | 10.0    | 860  | 0.3061          | 0.6303    | 0.6632 | 0.6463 | 0.9178   |
| 0.3332        | 11.0    | 946  | 0.2540          | 0.6569    | 0.7296 | 0.6913 | 0.9313   |
| 0.2715        | 12.0    | 1032 | 0.2007          | 0.7223    | 0.7707 | 0.7457 | 0.9461   |
| 0.2678        | 13.0    | 1118 | 0.1619          | 0.7506    | 0.8013 | 0.7751 | 0.9557   |
| 0.2267        | 14.0    | 1204 | 0.1468          | 0.7608    | 0.8318 | 0.7948 | 0.9603   |
| 0.1875        | 15.0    | 1290 | 0.1357          | 0.7759    | 0.8413 | 0.8073 | 0.9640   |
| 0.1753        | 16.0    | 1376 | 0.1166          | 0.8112    | 0.8651 | 0.8372 | 0.9692   |
| 0.1616        | 17.0    | 1462 | 0.0967          | 0.8204    | 0.8788 | 0.8486 | 0.9731   |
| 0.1337        | 18.0    | 1548 | 0.0854          | 0.8389    | 0.8951 | 0.8661 | 0.9762   |
| 0.1298        | 19.0    | 1634 | 0.0676          | 0.8623    | 0.9014 | 0.8814 | 0.9804   |
| 0.1115        | 20.0    | 1720 | 0.0701          | 0.8687    | 0.9135 | 0.8905 | 0.9808   |
| 0.1139        | 21.0    | 1806 | 0.0602          | 0.8916    | 0.9278 | 0.9093 | 0.9830   |
| 0.114         | 22.0    | 1892 | 0.0543          | 0.8957    | 0.9278 | 0.9114 | 0.9842   |
| 0.0944        | 23.0    | 1978 | 0.0569          | 0.8922    | 0.9341 | 0.9127 | 0.9843   |
| 0.0893        | 24.0    | 2064 | 0.0517          | 0.8986    | 0.9346 | 0.9163 | 0.9852   |
| 0.0836        | 25.0    | 2150 | 0.0476          | 0.9057    | 0.9367 | 0.9210 | 0.9862   |
| 0.0841        | 26.0    | 2236 | 0.0489          | 0.9062    | 0.9367 | 0.9212 | 0.9859   |
| 0.0865        | 27.0    | 2322 | 0.0459          | 0.9095    | 0.9378 | 0.9234 | 0.9866   |
| 0.0859        | 28.0    | 2408 | 0.0464          | 0.9096    | 0.9394 | 0.9243 | 0.9866   |
| 0.0796        | 29.0    | 2494 | 0.0461          | 0.9101    | 0.9394 | 0.9245 | 0.9866   |
| 0.0774        | 29.6550 | 2550 | 0.0460          | 0.9110    | 0.9394 | 0.9250 | 0.9866   |


### Framework versions

- Transformers 4.50.3
- Pytorch 2.6.0+cu124
- Datasets 3.5.0
- Tokenizers 0.21.1