readme: ---
  license: mit
  tags:
  - generated_from_trainer
  metrics:
  - f1
  - accuracy
  model-index:
  - name: xlm-sustainability-sentiment
    results: []
  ---
  
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
  should probably proofread and complete it, then remove this comment. -->
  
  # xlm-sustainability-sentiment
  
  This model is a fine-tuned version of [Raccourci/fairguest-bert](https://huggingface.co/Raccourci/fairguest-bert) on the None dataset.
  It achieves the following results on the evaluation set:
  - Loss: 0.2997
  - F1: 0.9335
  - Roc Auc: 0.9335
  - Accuracy: 0.9335
  
  ## Model description
  
  More information needed
  
  ## Intended uses & limitations
  
  More information needed
  
  ## Training and evaluation data
  
  More information needed
  
  ## Training procedure
  
  ### Training hyperparameters
  
  The following hyperparameters were used during training:
  - learning_rate: 2e-05
  - train_batch_size: 32
  - eval_batch_size: 32
  - seed: 42
  - gradient_accumulation_steps: 4
  - total_train_batch_size: 128
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  - lr_scheduler_type: linear
  - num_epochs: 20
  - mixed_precision_training: Native AMP
  
  ### Training results
  
  | Training Loss | Epoch | Step | Validation Loss | F1     | Roc Auc | Accuracy |
  |:-------------:|:-----:|:----:|:---------------:|:------:|:-------:|:--------:|
  | No log        | 0.98  | 15   | 0.5221          | 0.7173 | 0.7173  | 0.7173   |
  | No log        | 1.98  | 30   | 0.3833          | 0.7365 | 0.7308  | 0.7089   |
  | No log        | 2.98  | 45   | 0.3204          | 0.9030 | 0.9012  | 0.8836   |
  | No log        | 3.98  | 60   | 0.2861          | 0.8960 | 0.8960  | 0.8960   |
  | No log        | 4.98  | 75   | 0.2223          | 0.9125 | 0.9127  | 0.9106   |
  | No log        | 5.98  | 90   | 0.2499          | 0.9210 | 0.9210  | 0.9210   |
  | No log        | 6.98  | 105  | 0.2168          | 0.9293 | 0.9293  | 0.9293   |
  | No log        | 7.98  | 120  | 0.2122          | 0.9376 | 0.9376  | 0.9376   |
  | No log        | 8.98  | 135  | 0.2303          | 0.9335 | 0.9335  | 0.9335   |
  | No log        | 9.98  | 150  | 0.2455          | 0.9314 | 0.9314  | 0.9314   |
  | No log        | 10.98 | 165  | 0.2278          | 0.9335 | 0.9335  | 0.9335   |
  | No log        | 11.98 | 180  | 0.2593          | 0.9304 | 0.9304  | 0.9293   |
  | No log        | 12.98 | 195  | 0.2494          | 0.9397 | 0.9397  | 0.9397   |
  | No log        | 13.98 | 210  | 0.2579          | 0.9314 | 0.9314  | 0.9314   |
  | No log        | 14.98 | 225  | 0.2633          | 0.9356 | 0.9356  | 0.9356   |
  | No log        | 15.98 | 240  | 0.2918          | 0.9283 | 0.9283  | 0.9272   |
  | No log        | 16.98 | 255  | 0.2714          | 0.9356 | 0.9356  | 0.9356   |
  | No log        | 17.98 | 270  | 0.3034          | 0.9356 | 0.9356  | 0.9356   |
  | No log        | 18.98 | 285  | 0.3050          | 0.9325 | 0.9324  | 0.9314   |
  | No log        | 19.98 | 300  | 0.2997          | 0.9335 | 0.9335  | 0.9335   |
  
  
  ### Framework versions
  
  - Transformers 4.25.1
  - Pytorch 1.13.0+cu116
  - Datasets 2.8.0
  - Tokenizers 0.13.2
  