End of training

README.md

@@ -0,0 +1,101 @@
+---
+license: apache-2.0
+base_model: projecte-aina/roberta-base-ca-v2-cased-te
+tags:
+- generated_from_trainer
+metrics:
+- accuracy
+- precision
+- recall
+- f1
+model-index:
+- name: 080524_epoch_1
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# 080524_epoch_1
+
+This model is a fine-tuned version of [projecte-aina/roberta-base-ca-v2-cased-te](https://huggingface.co/projecte-aina/roberta-base-ca-v2-cased-te) on the None dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.3600
+- Accuracy: 0.969
+- Precision: 0.9692
+- Recall: 0.969
+- F1: 0.9690
+- Ratio: 0.489
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 2e-05
+- train_batch_size: 10
+- eval_batch_size: 2
+- seed: 47
+- gradient_accumulation_steps: 2
+- total_train_batch_size: 20
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- lr_scheduler_warmup_ratio: 0.06
+- lr_scheduler_warmup_steps: 4
+- num_epochs: 1
+- label_smoothing_factor: 0.1
+
+### Training results
+
+| Training Loss | Epoch  | Step | Validation Loss | Accuracy | Precision | Recall | F1     | Ratio |
+|:-------------:|:------:|:----:|:---------------:|:--------:|:---------:|:------:|:------:|:-----:|
+| 3.2945        | 0.0333 | 10   | 1.1037          | 0.725    | 0.7286    | 0.7250 | 0.7239 | 0.563 |
+| 0.9071        | 0.0667 | 20   | 0.7622          | 0.787    | 0.8184    | 0.7870 | 0.7816 | 0.343 |
+| 0.76          | 0.1    | 30   | 0.5554          | 0.904    | 0.9045    | 0.904  | 0.9040 | 0.482 |
+| 0.614         | 0.1333 | 40   | 0.5082          | 0.925    | 0.9253    | 0.925  | 0.9250 | 0.513 |
+| 0.5943        | 0.1667 | 50   | 0.4751          | 0.936    | 0.9376    | 0.9360 | 0.9359 | 0.47  |
+| 0.5569        | 0.2    | 60   | 0.4387          | 0.947    | 0.9472    | 0.9470 | 0.9470 | 0.489 |
+| 0.4397        | 0.2333 | 70   | 0.4339          | 0.937    | 0.9371    | 0.937  | 0.9370 | 0.491 |
+| 0.443         | 0.2667 | 80   | 0.4495          | 0.927    | 0.9299    | 0.927  | 0.9269 | 0.459 |
+| 0.5181        | 0.3    | 90   | 0.3937          | 0.949    | 0.9493    | 0.9490 | 0.9490 | 0.487 |
+| 0.4602        | 0.3333 | 100  | 0.4110          | 0.945    | 0.9467    | 0.9450 | 0.9449 | 0.531 |
+| 0.3992        | 0.3667 | 110  | 0.3943          | 0.951    | 0.9518    | 0.9510 | 0.9510 | 0.479 |
+| 0.3857        | 0.4    | 120  | 0.4006          | 0.951    | 0.9517    | 0.9510 | 0.9510 | 0.519 |
+| 0.4158        | 0.4333 | 130  | 0.3808          | 0.958    | 0.9581    | 0.958  | 0.9580 | 0.492 |
+| 0.422         | 0.4667 | 140  | 0.3809          | 0.956    | 0.9562    | 0.956  | 0.9560 | 0.49  |
+| 0.3816        | 0.5    | 150  | 0.3960          | 0.957    | 0.9573    | 0.957  | 0.9570 | 0.487 |
+| 0.4526        | 0.5333 | 160  | 0.3833          | 0.961    | 0.9610    | 0.961  | 0.9610 | 0.495 |
+| 0.4347        | 0.5667 | 170  | 0.3979          | 0.949    | 0.9510    | 0.9490 | 0.9489 | 0.467 |
+| 0.415         | 0.6    | 180  | 0.3673          | 0.963    | 0.9632    | 0.963  | 0.9630 | 0.491 |
+| 0.437         | 0.6333 | 190  | 0.3673          | 0.964    | 0.9640    | 0.964  | 0.9640 | 0.502 |
+| 0.3539        | 0.6667 | 200  | 0.3669          | 0.965    | 0.9650    | 0.965  | 0.9650 | 0.497 |
+| 0.4564        | 0.7    | 210  | 0.3713          | 0.964    | 0.9644    | 0.964  | 0.9640 | 0.486 |
+| 0.4008        | 0.7333 | 220  | 0.3639          | 0.966    | 0.9661    | 0.966  | 0.9660 | 0.492 |
+| 0.418         | 0.7667 | 230  | 0.3621          | 0.964    | 0.9640    | 0.964  | 0.9640 | 0.496 |
+| 0.4087        | 0.8    | 240  | 0.3581          | 0.968    | 0.9680    | 0.968  | 0.9680 | 0.498 |
+| 0.4132        | 0.8333 | 250  | 0.3579          | 0.967    | 0.9672    | 0.967  | 0.9670 | 0.489 |
+| 0.3861        | 0.8667 | 260  | 0.3622          | 0.967    | 0.9673    | 0.967  | 0.9670 | 0.487 |
+| 0.4345        | 0.9    | 270  | 0.3616          | 0.967    | 0.9673    | 0.967  | 0.9670 | 0.487 |
+| 0.3376        | 0.9333 | 280  | 0.3604          | 0.969    | 0.9692    | 0.969  | 0.9690 | 0.489 |
+| 0.4226        | 0.9667 | 290  | 0.3600          | 0.969    | 0.9692    | 0.969  | 0.9690 | 0.489 |
+| 0.3835        | 1.0    | 300  | 0.3600          | 0.969    | 0.9692    | 0.969  | 0.9690 | 0.489 |
+
+
+### Framework versions
+
+- Transformers 4.41.0
+- Pytorch 2.3.0+cu121
+- Datasets 2.19.1
+- Tokenizers 0.19.1

config.json

@@ -0,0 +1,40 @@
+{
+  "_name_or_path": "projecte-aina/roberta-base-ca-v2-cased-te",
+  "architectures": [
+    "RobertaForSequenceClassification"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "bos_token_id": 0,
+  "classifier_dropout": null,
+  "eos_token_id": 2,
+  "finetuning_task": "mnli",
+  "gradient_checkpointing": false,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "id2label": {
+    "0": "ENTAILMENT",
+    "1": "NEUTRAL",
+    "2": "CONTRADICTION"
+  },
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "label2id": {
+    "CONTRADICTION": 2,
+    "ENTAILMENT": 0,
+    "NEUTRAL": 1
+  },
+  "layer_norm_eps": 1e-05,
+  "max_position_embeddings": 514,
+  "model_type": "roberta",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pad_token_id": 1,
+  "position_embedding_type": "absolute",
+  "problem_type": "single_label_classification",
+  "torch_dtype": "float32",
+  "transformers_version": "4.41.0",
+  "type_vocab_size": 1,
+  "use_cache": true,
+  "vocab_size": 50262
+}

logs/events.out.tfevents.1716888931.a5a87f3fc98d.2577.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6d6d14dcdee7e2c811e66f4ec24e2cb1710654ba760dcac0c4ac372ff9fb5622
+size 27465

logs/events.out.tfevents.1716889807.a5a87f3fc98d.2577.1

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:34d98044468660755fa57f4854f39e9a56459e7cb0d43d0c97a4c564e7e615fa
+size 28014

logs/events.out.tfevents.1716890847.a5a87f3fc98d.2577.3

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6481e84b7ac9624274a6acb1a1d4b718e06ef5b8647d9f5b95bb42e92ae777bc
+size 27465

logs/events.out.tfevents.1716891723.a5a87f3fc98d.2577.4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1bb540e8ecadd2f6f3cedfa1a87ffafd8f9866c58263143a032e524c44e0c6e0
+size 609

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ed25ebc0de1c925f88483578d3e736c48baae5c50dc1be553e90cdb97bf2e480
+size 498606684

special_tokens_map.json

@@ -0,0 +1,51 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "cls_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "<mask>",
+    "lstrip": true,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<pad>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,65 @@
+{
+  "add_prefix_space": true,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<pad>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "4": {
+      "content": "<mask>",
+      "lstrip": true,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<s>",
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "<s>",
+  "eos_token": "</s>",
+  "errors": "replace",
+  "mask_token": "<mask>",
+  "max_len": 512,
+  "max_length": 512,
+  "model_max_length": 512,
+  "pad_to_multiple_of": null,
+  "pad_token": "<pad>",
+  "pad_token_type_id": 0,
+  "padding_side": "right",
+  "sep_token": "</s>",
+  "stride": 0,
+  "tokenizer_class": "RobertaTokenizer",
+  "trim_offsets": true,
+  "truncation_side": "right",
+  "truncation_strategy": "longest_first",
+  "unk_token": "<unk>"
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f1c24c5f75828526fdb58ea86eb1646744a8f5c63474fdba7874c2f00ae62083
+size 5112