transformers pytorch safetensors bert text-classification de license:other autotrain_compatible endpoints_compatible region:us
---
language: de
license: other
widget:
- text: Guten morgen, meine Liebe
  example_title: NOT TOXIC 1
- text: Ich scheiß drauf.
  example_title: TOXIC 1
- text: Ich liebe dich
  example_title: NOT TOXIC 2
- text: Ich hab die Schnauze voll von diesen Irren.
  example_title: TOXIC 2
- text: Ich wünsche Ihnen einen schönen Tag!
  example_title: NOT TOXIC 3
- text: Nigger
  example_title: TOXIC 3
- text: Du bist schon wieder zu spät!
  example_title: NOT TOXIC 4
- text: Beweg deinen AArschhh hier rüber
  example_title: TOXIC 4
---
## Description
NB: this version of the model is the improved version of [EIStakovskii/german_toxicity_classifier_plus](https://huggingface.co/EIStakovskii/german_toxicity_classifier_plus).
To see the source code of training and the data please follow [the github link](https://github.com/eistakovskii/NLP_projects/tree/main/TEXT_CLASSIFICATION).

This model was trained for toxicity labeling.

The model was fine-tuned based off [the dbmdz/bert-base-german-cased model](https://huggingface.co/dbmdz/bert-base-german-cased).

To use the model:

```python
from transformers import pipeline

classifier = pipeline("text-classification", model = 'EIStakovskii/german_toxicity_classifier_plus_v2')

print(classifier("Verpiss dich von hier"))

```

## Metrics (at validation):

epoch|step|eval_accuracy|eval_f1|eval_loss
-|-|-|-|-
0.8|1200|0.9132176234979973|0.9113535629048755|0.24135465919971466

## Comparison against Perspective

This model was compared against the Google's [Perspective API](https://developers.perspectiveapi.com/s/?language=en_US) that similarly detects toxicity. 
Two models were tested on two datasets: the size of [200 sentences](https://github.com/eistakovskii/NLP_Toolbox/blob/main/TEXT_CLASSIFICATION/Transformer_classifier/data/Toxicity_Classifiers/DE_FR/test/test_de_200.csv) and [400 sentences](https://github.com/eistakovskii/NLP_Toolbox/blob/main/TEXT_CLASSIFICATION/Transformer_classifier/data/Toxicity_Classifiers/DE_FR/test/test_de_400.csv). 
The first one (arguably harder) was collected from the sentences of the [JigSaw](https://www.kaggle.com/c/jigsaw-multilingual-toxic-comment-classification/data) and [DeTox](https://github.com/hdaSprachtechnologie/detox) datasets.
The second one (easier) was collected from the combination of sources: both from JigSaw and DeTox as well as [Paradetox](https://github.com/s-nlp/multilingual_detox/tree/main/data) translations and sentences extracted from [Reverso Context](https://context.reverso.net/translation/) by keywords.

# german_toxicity_classifier_plus_v2
size|accuracy|f1 
-|-|-
200|0.767|0.787
400|0.9650|0.9651

# Perspective
size|accuracy|f1 
-|-|-
200|0.834|0.820
400|0.892|0.885