LLM in the Loop: Creating the PARADEHATE Dataset for Hate Speech
  Detoxification
Detoxification, the task of rewriting harmful language into non-toxic text,
has become increasingly important amid the growing prevalence of toxic content
online. However, high-quality parallel datasets for detoxification, especially
for hate speech, remain scarce due to the cost and sensitivity of human
annotation. In this paper, we propose a novel LLM-in-the-loop pipeline
leveraging GPT-4o-mini for automated detoxification. We first replicate the
ParaDetox pipeline by replacing human annotators with an LLM and show that the
LLM performs comparably to human annotation. Building on this, we construct
PARADEHATE, a large-scale parallel dataset specifically for hatespeech
detoxification. We release PARADEHATE as a benchmark of over 8K hate/non-hate
text pairs and evaluate a wide range of baseline methods. Experimental results
show that models such as BART, fine-tuned on PARADEHATE, achieve better
performance in style accuracy, content preservation, and fluency, demonstrating
the effectiveness of LLM-generated detoxification text as a scalable
alternative to human annotation.
683eb167b3f3b41729e1d2e8
Shuzhou Yuan
claimed_verified
2025-06-03T10:45:13.850Z
662ce44c8b8705f30371fba8
/avatars/b96a25a8c124e7caa9de06b7188bdc15.svg
Shuzhou Yuan
user
shuzyuan
683eb167b3f3b41729e1d2e9
Ercong Nie
683eb167b3f3b41729e1d2ea
Lukas Kouba
683eb167b3f3b41729e1d2eb
Ashish Yashwanth Kangen
683eb167b3f3b41729e1d2ec
Helmut Schmid
683eb167b3f3b41729e1d2ed
Hinrich Schutze
683eb167b3f3b41729e1d2ee
Michael Farber
LLM in the Loop: Creating the PARADEHATE Dataset for Hate Speech
  Detoxification
Detoxification, the task of rewriting harmful language into non-toxic text,
has become increasingly important amid the growing prevalence of toxic content
online. However, high-quality parallel datasets for detoxification, especially
for hate speech, remain scarce due to the cost and sensitivity of human
annotation. In this paper, we propose a novel LLM-in-the-loop pipeline
leveraging GPT-4o-mini for automated detoxification. We first replicate the
ParaDetox pipeline by replacing human annotators with an LLM and show that the
LLM performs comparably to human annotation. Building on this, we construct
PARADEHATE, a large-scale parallel dataset specifically for hatespeech
detoxification. We release PARADEHATE as a benchmark of over 8K hate/non-hate
text pairs and evaluate a wide range of baseline methods. Experimental results
show that models such as BART, fine-tuned on PARADEHATE, achieve better
performance in style accuracy, content preservation, and fluency, demonstrating
the effectiveness of LLM-generated detoxification text as a scalable
alternative to human annotation.
A novel pipeline using GPT-4o-mini generates a large-scale dataset for hate speech detoxification, improving baseline model performance in style accuracy, content preservation, and fluency.
LLM-in-the-loop GPT-4o-mini ParaDetox PARADEHATE BART style accuracy content preservation fluency