readme: ---
  license: mit
  metrics:
  - accuracy
  - precision
  - recall
  model-index:
  - name: ManiBERT
    results: []
  widget: 
  - text: "Russia must end the war."
  - text: "Democratic institutions must be supported."
  - text: "The state must fight political corruption."
  - text: "Our energy economy must be nationalised."
  - text: "We must increase social spending."
  
  ---
  
  
  # ManiBERT
  
  This model is a fine-tuned version of [roberta-base](https://huggingface.co/roberta-base) on data from the [Manifesto Project](https://manifesto-project.wzb.eu/).
  
  
  ## Model description
  
  This model was trained on 115,943 manually annotated sentences to classify text into one of 56 political categories:
  
  
  ## Intended uses & limitations
  
  The model output reproduces the limitations of the dataset in terms of country coverage, time span, domain definitions and potential biases of the annotators - as any supervised machine learning model would. Applying the model to other types of data (other types of texts, countries etc.) will reduce performance.
  
  ```python
  from transformers import pipeline
  import pandas as pd
  classifier = pipeline(
      task="text-classification",
      model="niksmer/ManiBERT")
  # Load text data you want to classify
  text = pd.read_csv("example.csv")["text_you_want_to_classify"].to_list()
  # Inference
  output = classifier(text)
  # Print output
  pd.DataFrame(output).head()
  ```
  
  ## Train Data
  
  ManiBERT was trained on the English-speaking subset of the [Manifesto Project Dataset (MPDS2021a)](https://manifesto-project.wzb.eu/datasets). The model was trained on 115,943 sentences from 163 political manifestos in 7 English-speaking countries (Australia, Canada, Ireland, New Zealand, South Africa, United Kingdom, United States). The manifestos were published between 1992 - 2020. 
  
  
  | Country        | Count manifestos | Count sentences | Time span        |
  |----------------|------------------|-----------------|--------------------|
  | Australia      | 18               | 14,887          | 2010-2016          |
  | Ireland        | 23               | 24,966          | 2007-2016          |
  | Canada         | 14               | 12,344          | 2004-2008 & 2015   |
  | New Zealand    | 46               | 35,079          | 1993-2017          |
  | South Africa   | 29               | 13,334          | 1994-2019          |
  | USA            | 9                | 13,188          | 1992   & 2004-2020 |
  | United Kingdom | 34               | 30,936          | 1997-2019          |
  
  Canadian manifestos between 2004 and 2008 are used as test data.
  
  The resulting Datasets are higly (!) imbalanced. See Evaluation.
  
  ## Evaluation
  
  
  | Description                                                       | Label | Count Train Data | Count Validation Data | Count Test Data | Validation F1-Score | Test F1-Score |
  |-------------------------------------------------------------------|-------|------------------|-----------------------|-----------------|---------------------|---------------|
  | Foreign   Special Relationships: Positive                         | 0     | 545              | 96                    | 60              | 0.43                | 0.45          |
  | Foreign   Special Relationships: Negative                         | 1     | 66               | 14                    | 22              | 0.22                | 0.09          |
  | Anti-Imperialism                                                  | 2     | 93               | 16                    | 1               | 0.16                | 0.00          |
  | Military:   Positive                                              | 3     | 1,969            | 356                   | 159             | 0.69                | 0.63          |
  | Military:   Negative                                              | 4     | 489              | 89                    | 52              | 0.59                | 0.63          |
  | Peace                                                             | 5     | 418              | 80                    | 49              | 0.57                | 0.64          |
  | Internationalism:   Positive                                      | 6     | 2,401            | 417                   | 404             | 0.60                | 0.54          |
  | European   Community/Union or Latin America Integration: Positive | 7     | 930              | 156                   | 20              | 0.58                | 0.32          |
  | Internationalism:   Negative                                      | 8     | 209              | 40                    | 57              | 0.28                | 0.05          |
  | European   Community/Union or Latin America Integration: Negative | 9     | 520              | 81                    | 0               | 0.39                | -             |
  | Freedom   and Human Rights                                        | 10    | 2,196            | 389                   | 76              | 0.50                | 0.34          |
  | Democracy                                                         | 11    | 3,045      