readme: 
        
  <div align="center">    
  
  **⚠️ Disclaimer:**
  The huggingface models currently give different results to the detoxify library (see issue [here](https://github.com/unitaryai/detoxify/issues/15)). For the most up to date models we recommend using the models from https://github.com/unitaryai/detoxify
  
  # 🙊 Detoxify
  ##  Toxic Comment Classification with ⚡ Pytorch Lightning and 🤗 Transformers   
  
  ![CI testing](https://github.com/unitaryai/detoxify/workflows/CI%20testing/badge.svg)
  ![Lint](https://github.com/unitaryai/detoxify/workflows/Lint/badge.svg)
  
  </div>
   
  ![Examples image](examples.png)
  
  ## Description   
  
  Trained models & code to predict toxic comments on 3 Jigsaw challenges: Toxic comment classification, Unintended Bias in Toxic comments, Multilingual toxic comment classification.
  
  Built by [Laura Hanu](https://laurahanu.github.io/) at [Unitary](https://www.unitary.ai/), where we are working to stop harmful content online by interpreting visual content in context. 
  
  Dependencies:
  - For inference:
    - 🤗 Transformers
    - ⚡ Pytorch lightning 
  - For training will also need:
    - Kaggle API (to download data)
  
  
  | Challenge | Year | Goal | Original Data Source | Detoxify Model Name | Top Kaggle Leaderboard Score | Detoxify Score
  |-|-|-|-|-|-|-|
  | [Toxic Comment Classification Challenge](https://www.kaggle.com/c/jigsaw-toxic-comment-classification-challenge) | 2018 |  build a multi-headed model that’s capable of detecting different types of of toxicity like threats, obscenity, insults, and identity-based hate. | Wikipedia Comments | `original` | 0.98856 | 0.98636
  | [Jigsaw Unintended Bias in Toxicity Classification](https://www.kaggle.com/c/jigsaw-unintended-bias-in-toxicity-classification) | 2019 | build a model that recognizes toxicity and minimizes this type of unintended bias with respect to mentions of identities. You'll be using a dataset labeled for identity mentions and optimizing a metric designed to measure unintended bias. | Civil Comments | `unbiased` | 0.94734 | 0.93639
  | [Jigsaw Multilingual Toxic Comment Classification](https://www.kaggle.com/c/jigsaw-multilingual-toxic-comment-classification) | 2020 | build effective multilingual models | Wikipedia Comments + Civil Comments | `multilingual` | 0.9536 | 0.91655*
  
  *Score not directly comparable since it is obtained on the validation set provided and not on the test set. To update when the test labels are made available. 
  
  It is also noteworthy to mention that the top leadearboard scores have been achieved using model ensembles. The purpose of this library was to build something user-friendly and straightforward to use.
  
  ## Limitations and ethical considerations
  
  If words that are associated with swearing, insults or profanity are present in a comment, it is likely that it will be classified as toxic, regardless of the tone or the intent of the author e.g. humorous/self-deprecating. This could present some biases towards already vulnerable minority groups.
  
  The intended use of this library is for research purposes, fine-tuning on carefully constructed datasets that reflect real world demographics  and/or to aid content moderators in flagging out harmful content quicker.
  
  Some useful resources about the risk of different biases in toxicity or hate speech detection are:
  - [The Risk of Racial Bias in Hate Speech Detection](https://homes.cs.washington.edu/~msap/pdfs/sap2019risk.pdf)
  - [Automated Hate Speech Detection and the Problem of Offensive Language](https://arxiv.org/pdf/1703.04009.pdf%201.pdf)
  - [Racial Bias in Hate Speech and Abusive Language Detection Datasets](https://arxiv.org/pdf/1905.12516.pdf)
  
  ## Quick prediction
  
  
  The `multilingual` model has been trained on 7 different languages so it should only be tested on: `english`, `french`, `spanish`, `italian`, `portuguese`, `turkish` or `russian`.
  
  ```bash
  # install detoxify  
  
  pip install detoxify
  
  ```
  ```python
  
  from detoxify import Detoxify
  
  # each model takes in either a string or a list of strings
  
  results = Detoxify('original').predict('example text')
  
  results = Detoxify('unbiased').predict(['example text 1','example text 2'])
  
  results = Detoxify('multilingual').predict(['example text','exemple de texte','texto de ejemplo','testo di esempio','texto de exemplo','örnek metin','пример текста'])
  
  # optional to display results nicely (will need to pip install pandas)
  
  import pandas as pd
  
  print(pd.DataFrame(results, index=input_text).round(5))
  
  ```
  For more details check the Prediction section.
  
  
  ## Labels
  All challenges have a toxicity label. The toxicity labels represent the aggregate ratings of up to 10 annotators according the following schema:
  - **Very Toxic** (a very hateful, aggressive, or disrespectful comment that is very likely to make you leave a discussion or give up on sharing your perspective)
  - **Toxic** (a rude, disrespectful, or unreasonable comment that is somewhat likely to make you leave a discussion or give up on sharing your perspective)
  - **Hard to Say**
  - **Not Tox