transformers safetensors bert text-classification generated_from_trainer dataset:sem_eval_2024_task_2 base_model:MoritzLaurer/DeBERTa-v3-base-mnli-fever-anli base_model:finetune:MoritzLaurer/DeBERTa-v3-base-mnli-fever-anli license:mit model-index autotrain_compatible endpoints_compatible region:us
---
base_model: MoritzLaurer/DeBERTa-v3-base-mnli-fever-anli
datasets:
- sem_eval_2024_task_2
license: mit
metrics:
- accuracy
- precision
- recall
- f1
tags:
- generated_from_trainer
model-index:
- name: results2
  results:
  - task:
      type: text-classification
      name: Text Classification
    dataset:
      name: sem_eval_2024_task_2
      type: sem_eval_2024_task_2
      config: sem_eval_2024_task_2_source
      split: validation
      args: sem_eval_2024_task_2_source
    metrics:
    - type: accuracy
      value: 0.76
      name: Accuracy
    - type: precision
      value: 0.7601040416166467
      name: Precision
    - type: recall
      value: 0.76
      name: Recall
    - type: f1
      value: 0.75997599759976
      name: F1
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# results2

This model is a fine-tuned version of [MoritzLaurer/DeBERTa-v3-base-mnli-fever-anli](https://huggingface.co/MoritzLaurer/DeBERTa-v3-base-mnli-fever-anli) on the sem_eval_2024_task_2 dataset.
It achieves the following results on the evaluation set:
- Loss: 2.1827
- Accuracy: 0.76
- Precision: 0.7601
- Recall: 0.76
- F1: 0.7600

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- num_epochs: 50
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step | Validation Loss | Accuracy | Precision | Recall | F1     |
|:-------------:|:-----:|:----:|:---------------:|:--------:|:---------:|:------:|:------:|
| 0.6925        | 1.0   | 107  | 0.6665          | 0.6      | 0.6457    | 0.6    | 0.5660 |
| 0.6729        | 2.0   | 214  | 0.6025          | 0.69     | 0.6964    | 0.69   | 0.6875 |
| 0.6857        | 3.0   | 321  | 0.6071          | 0.665    | 0.7531    | 0.665  | 0.6331 |
| 0.6667        | 4.0   | 428  | 0.5650          | 0.695    | 0.7157    | 0.6950 | 0.6875 |
| 0.6168        | 5.0   | 535  | 0.5036          | 0.75     | 0.7504    | 0.75   | 0.7499 |
| 0.5165        | 6.0   | 642  | 0.6248          | 0.67     | 0.6701    | 0.67   | 0.6700 |
| 0.4087        | 7.0   | 749  | 0.5246          | 0.735    | 0.7379    | 0.7350 | 0.7342 |
| 0.3083        | 8.0   | 856  | 0.6130          | 0.7      | 0.7       | 0.7    | 0.7    |
| 0.2909        | 9.0   | 963  | 0.7584          | 0.735    | 0.7723    | 0.7350 | 0.7256 |
| 0.319         | 10.0  | 1070 | 0.7350          | 0.72     | 0.7360    | 0.72   | 0.7152 |
| 0.1812        | 11.0  | 1177 | 0.9320          | 0.715    | 0.7176    | 0.7150 | 0.7141 |
| 0.2824        | 12.0  | 1284 | 0.9723          | 0.705    | 0.7336    | 0.7050 | 0.6957 |
| 0.2662        | 13.0  | 1391 | 0.8676          | 0.72     | 0.7222    | 0.72   | 0.7193 |
| 0.1641        | 14.0  | 1498 | 0.9450          | 0.71     | 0.7103    | 0.71   | 0.7099 |
| 0.2264        | 15.0  | 1605 | 1.1613          | 0.675    | 0.6764    | 0.675  | 0.6743 |
| 0.2077        | 16.0  | 1712 | 1.3497          | 0.715    | 0.7214    | 0.7150 | 0.7129 |
| 0.1767        | 17.0  | 1819 | 1.4154          | 0.705    | 0.7075    | 0.7050 | 0.7041 |
| 0.1751        | 18.0  | 1926 | 1.2369          | 0.735    | 0.7350    | 0.735  | 0.7350 |
| 0.1195        | 19.0  | 2033 | 1.1152          | 0.72     | 0.7334    | 0.72   | 0.7159 |
| 0.0507        | 20.0  | 2140 | 1.4853          | 0.715    | 0.7152    | 0.715  | 0.7149 |
| 0.0544        | 21.0  | 2247 | 1.7174          | 0.725    | 0.7302    | 0.7250 | 0.7234 |
| 0.0648        | 22.0  | 2354 | 1.7327          | 0.71     | 0.7121    | 0.71   | 0.7093 |
| 0.0039        | 23.0  | 2461 | 1.8211          | 0.725    | 0.7268    | 0.7250 | 0.7244 |
| 0.0153        | 24.0  | 2568 | 1.8315          | 0.715    | 0.7176    | 0.7150 | 0.7141 |
| 0.0017        | 25.0  | 2675 | 1.7446          | 0.72     | 0.7232    | 0.72   | 0.7190 |
| 0.0188        | 26.0  | 2782 | 1.6413          | 0.72     | 0.7274    | 0.72   | 0.7177 |
| 0.0168        | 27.0  | 2889 | 1.8013          | 0.73     | 0.7315    | 0.73   | 0.7296 |
| 0.0355        | 28.0  | 2996 | 2.0405          | 0.725    | 0.7354    | 0.725  | 0.7219 |
| 0.0168        | 29.0  | 3103 | 1.5087          | 0.735    | 0.7350    | 0.735  | 0.7350 |
| 0.0409        | 30.0  | 3210 | 1.5272          | 0.72     | 0.7244    | 0.72   | 0.7186 |
| 0.004         | 31.0  | 3317 | 1.9978          | 0.715    | 0.7214    | 0.7150 | 0.7129 |
| 0.0002        | 32.0  | 3424 | 1.9760          | 0.72     | 0.7244    | 0.72   | 0.7186 |
| 0.0111        | 33.0  | 3531 | 1.9985          | 0.74     | 0.7409    | 0.74   | 0.7398 |
| 0.052         | 34.0  | 3638 | 1.9607          | 0.73     | 0.7334    | 0.73   | 0.7290 |
| 0.0263        | 35.0  | 3745 | 1.7118          | 0.75     | 0.7525    | 0.75   | 0.7494 |
| 0.0101        | 36.0  | 3852 | 1.9553          | 0.755    | 0.7571    | 0.755  | 0.7545 |
| 0.0001        | 37.0  | 3959 | 2.0064          | 0.75     | 0.7537    | 0.75   | 0.7491 |
| 0.0186        | 38.0  | 4066 | 2.1726          | 0.74     | 0.7404    | 0.74   | 0.7399 |
| 0.0046        | 39.0  | 4173 | 2.1083          | 0.755    | 0.7550    | 0.755  | 0.7550 |
| 0.0042        | 40.0  | 4280 | 1.9944          | 0.76     | 0.7609    | 0.76   | 0.7598 |
| 0.0178        | 41.0  | 4387 | 2.0096          | 0.76     | 0.7604    | 0.76   | 0.7599 |
| 0.0089        | 42.0  | 4494 | 2.0431          | 0.765    | 0.7652    | 0.765  | 0.7649 |
| 0.0095        | 43.0  | 4601 | 2.0662          | 0.76     | 0.7604    | 0.76   | 0.7599 |
| 0.0162        | 44.0  | 4708 | 2.1703          | 0.745    | 0.7450    | 0.745  | 0.7450 |
| 0.0001        | 45.0  | 4815 | 2.1525          | 0.76     | 0.7601    | 0.76   | 0.7600 |
| 0.0001        | 46.0  | 4922 | 2.1581          | 0.76     | 0.7601    | 0.76   | 0.7600 |
| 0.0086        | 47.0  | 5029 | 2.1665          | 0.76     | 0.7601    | 0.76   | 0.7600 |
| 0.0088        | 48.0  | 5136 | 2.1747          | 0.76     | 0.7601    | 0.76   | 0.7600 |
| 0.0044        | 49.0  | 5243 | 2.1812          | 0.76     | 0.7601    | 0.76   | 0.7600 |
| 0.0043        | 50.0  | 5350 | 2.1827          | 0.76     | 0.7601    | 0.76   | 0.7600 |


### Framework versions

- Transformers 4.35.2
- Pytorch 2.1.0+cu118
- Datasets 2.15.0
- Tokenizers 0.15.0
