readme: ---
  license: mit
  language:
  - am
  - ar
  - hy
  - eu
  - bn
  - bs
  - bg
  - my
  - hr
  - ca
  - cs
  - da
  - nl
  - en
  - et
  - fi
  - fr
  - ka
  - de
  - el
  - gu
  - ht
  - iw
  - hi
  - hu
  - is
  - in
  - it
  - ja
  - kn
  - km
  - ko
  - lo
  - lv
  - lt
  - ml
  - mr
  - ne
  - no
  - or
  - pa
  - ps
  - fa
  - pl
  - pt
  - ro
  - ru
  - sr
  - zh
  - sd
  - si
  - sk
  - sl
  - es
  - sv
  - tl
  - ta
  - te
  - th
  - tr
  - uk
  - ur
  - ug
  - vi
  - cy
  tags:
  - generated_from_trainer
  model-index:
  - name: verdict-classifier-en
    results:
    - task:
        type: text-classification
        name: Verdict Classification
  widget:
  - "本文已断章取义。"
  ---
  
  # Multilingual Verdict Classifier
  
  This model is a fine-tuned version of [xlm-roberta-base](https://huggingface.co/xlm-roberta-base) on 2,500 deduplicated multilingual verdicts from [Google Fact Check Tools API](https://developers.google.com/fact-check/tools/api/reference/rest/v1alpha1/claims/search), translated into 65 languages with the [Google Cloud Translation API](https://cloud.google.com/translate/docs/reference/rest/).
  It achieves the following results on the evaluation set, being 1,000 such verdicts, but here including duplicates to represent the true distribution:
  - Loss: 0.2238
  - F1 Macro: 0.8540
  - F1 Misinformation: 0.9798
  - F1 Factual: 0.9889
  - F1 Other: 0.5934
  - Prec Macro: 0.8348
  - Prec Misinformation: 0.9860
  - Prec Factual: 0.9889
  - Prec Other: 0.5294
  
  
  ## Training procedure
  
  ### Training hyperparameters
  
  The following hyperparameters were used during training:
  - learning_rate: 2e-05
  - train_batch_size: 4
  - eval_batch_size: 4
  - seed: 42
  - gradient_accumulation_steps: 8
  - total_train_batch_size: 32
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  - lr_scheduler_type: linear
  - lr_scheduler_warmup_steps: 162525
  - num_epochs: 1000
  
  ### Training results
  
  | Training Loss | Epoch | Step  | Validation Loss | F1 Macro | F1 Misinformation | F1 Factual | F1 Other | Prec Macro | Prec Misinformation | Prec Factual | Prec Other |
  |:-------------:|:-----:|:-----:|:---------------:|:--------:|:-----------------:|:----------:|:--------:|:----------:|:-------------------:|:------------:|:----------:|
  | 1.1109        | 0.1   | 2000  | 1.2166          | 0.0713   | 0.1497            | 0.0        | 0.0640   | 0.2451     | 0.7019              | 0.0          | 0.0334     |
  | 0.9551        | 0.2   | 4000  | 0.7801          | 0.3611   | 0.8889            | 0.0        | 0.1943   | 0.3391     | 0.8915              | 0.0          | 0.1259     |
  | 0.9275        | 0.3   | 6000  | 0.7712          | 0.3468   | 0.9123            | 0.0        | 0.1282   | 0.3304     | 0.9051              | 0.0          | 0.0862     |
  | 0.8881        | 0.39  | 8000  | 0.5386          | 0.3940   | 0.9524            | 0.0        | 0.2297   | 0.3723     | 0.9748              | 0.0          | 0.1420     |
  | 0.7851        | 0.49  | 10000 | 0.3298          | 0.6886   | 0.9626            | 0.7640     | 0.3393   | 0.6721     | 0.9798              | 0.7727       | 0.2639     |
  | 0.639         | 0.59  | 12000 | 0.2156          | 0.7847   | 0.9633            | 0.9355     | 0.4554   | 0.7540     | 0.9787              | 0.9062       | 0.3770     |
  | 0.5677        | 0.69  | 14000 | 0.1682          | 0.7877   | 0.9694            | 0.9667     | 0.4270   | 0.7763     | 0.9745              | 0.9667       | 0.3878     |
  | 0.5218        | 0.79  | 16000 | 0.1475          | 0.8037   | 0.9692            | 0.9667     | 0.4752   | 0.7804     | 0.9812              | 0.9667       | 0.3934     |
  | 0.4682        | 0.89  | 18000 | 0.1458          | 0.8097   | 0.9734            | 0.9667     | 0.4889   | 0.7953     | 0.9791              | 0.9667       | 0.44       |
  | 0.4188        | 0.98  | 20000 | 0.1416          | 0.8370   | 0.9769            | 0.9724     | 0.5618   | 0.8199     | 0.9826              | 0.9670       | 0.5102     |
  | 0.3735        | 1.08  | 22000 | 0.1624          | 0.8094   | 0.9698            | 0.9368     | 0.5217   | 0.7780     | 0.9823              | 0.89         | 0.4615     |
  | 0.3242        | 1.18  | 24000 | 0.1648          | 0.8338   | 0.9769            | 0.9727     | 0.5517   | 0.8167     | 0.9826              | 0.9570       | 0.5106     |
  | 0.2785        | 1.28  | 26000 | 0.1843          | 0.8261   | 0.9739            | 0.9780     | 0.5263   | 0.8018     | 0.9836              | 0.9674       | 0.4545     |
  | 0.25          | 1.38  | 28000 | 0.1975          | 0.8344   | 0.9744            | 0.9834     | 0.5455   | 0.8072     | 0.9859              | 0.9780       | 0.4576     |
  | 0.2176        | 1.48  | 30000 | 0.1849          | 0.8209   | 0.9691            | 0.9889     | 0.5047   | 0.7922     | 0.9846              | 0.9889       | 0.4030     |
  | 0.1966        | 1.58  | 32000 | 0.2119          | 0.8194   | 0.9685            | 0.9944     | 0.4954   | 0.7920     | 0.9846              | 1.0          | 0.3913     |
  | 0.1738        | 1.67  | 34000 | 0.2110          | 0.8352   | 0.9708            | 0.9944     | 0.5405   | 0.8035     | 0.9881              | 1.0          | 0.4225     |
  | 0.1625        | 1.77  | 36000 | 0.2