title: LLM in the Loop: Creating the PARADEHATE Dataset for Hate Speech
    Detoxification
summary: Detoxification, the task of rewriting harmful language into non-toxic text,
  has become increasingly important amid the growing prevalence of toxic content
  online. However, high-quality parallel datasets for detoxification, especially
  for hate speech, remain scarce due to the cost and sensitivity of human
  annotation. In this paper, we propose a novel LLM-in-the-loop pipeline
  leveraging GPT-4o-mini for automated detoxification. We first replicate the
  ParaDetox pipeline by replacing human annotators with an LLM and show that the
  LLM performs comparably to human annotation. Building on this, we construct
  PARADEHATE, a large-scale parallel dataset specifically for hatespeech
  detoxification. We release PARADEHATE as a benchmark of over 8K hate/non-hate
  text pairs and evaluate a wide range of baseline methods. Experimental results
  show that models such as BART, fine-tuned on PARADEHATE, achieve better
  performance in style accuracy, content preservation, and fluency, demonstrating
  the effectiveness of LLM-generated detoxification text as a scalable
  alternative to human annotation.
paper_authors._id: 683eb167b3f3b41729e1d2e8
paper_authors.name: Shuzhou Yuan
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-03T10:45:13.850Z
paper_authors._id: 662ce44c8b8705f30371fba8
paper_authors.avatarUrl: /avatars/b96a25a8c124e7caa9de06b7188bdc15.svg
paper_authors.fullname: Shuzhou Yuan
paper_authors.type: user
paper_authors.user: shuzyuan
paper_authors._id: 683eb167b3f3b41729e1d2e9
paper_authors.name: Ercong Nie
paper_authors._id: 683eb167b3f3b41729e1d2ea
paper_authors.name: Lukas Kouba
paper_authors._id: 683eb167b3f3b41729e1d2eb
paper_authors.name: Ashish Yashwanth Kangen
paper_authors._id: 683eb167b3f3b41729e1d2ec
paper_authors.name: Helmut Schmid
paper_authors._id: 683eb167b3f3b41729e1d2ed
paper_authors.name: Hinrich Schutze
paper_authors._id: 683eb167b3f3b41729e1d2ee
paper_authors.name: Michael Farber
paper_title: LLM in the Loop: Creating the PARADEHATE Dataset for Hate Speech
    Detoxification
paper_summary: Detoxification, the task of rewriting harmful language into non-toxic text,
  has become increasingly important amid the growing prevalence of toxic content
  online. However, high-quality parallel datasets for detoxification, especially
  for hate speech, remain scarce due to the cost and sensitivity of human
  annotation. In this paper, we propose a novel LLM-in-the-loop pipeline
  leveraging GPT-4o-mini for automated detoxification. We first replicate the
  ParaDetox pipeline by replacing human annotators with an LLM and show that the
  LLM performs comparably to human annotation. Building on this, we construct
  PARADEHATE, a large-scale parallel dataset specifically for hatespeech
  detoxification. We release PARADEHATE as a benchmark of over 8K hate/non-hate
  text pairs and evaluate a wide range of baseline methods. Experimental results
  show that models such as BART, fine-tuned on PARADEHATE, achieve better
  performance in style accuracy, content preservation, and fluency, demonstrating
  the effectiveness of LLM-generated detoxification text as a scalable
  alternative to human annotation.
paper_ai_summary: A novel pipeline using GPT-4o-mini generates a large-scale dataset for hate speech detoxification, improving baseline model performance in style accuracy, content preservation, and fluency.
paper_ai_keywords: LLM-in-the-loop GPT-4o-mini ParaDetox PARADEHATE BART style accuracy content preservation fluency