title: Understanding and Mitigating Toxicity in Image-Text Pretraining
    Datasets: A Case Study on LLaVA
summary: Pretraining datasets are foundational to the development of multimodal
  models, yet they often have inherent biases and toxic content from the
  web-scale corpora they are sourced from. In this paper, we investigate the
  prevalence of toxicity in LLaVA image-text pretraining dataset, examining how
  harmful content manifests in different modalities. We present a comprehensive
  analysis of common toxicity categories and propose targeted mitigation
  strategies, resulting in the creation of a refined toxicity-mitigated dataset.
  This dataset removes 7,531 of toxic image-text pairs in the LLaVA pre-training
  dataset. We offer guidelines for implementing robust toxicity detection
  pipelines. Our findings underscore the need to actively identify and filter
  toxic content - such as hate speech, explicit imagery, and targeted harassment
  - to build more responsible and equitable multimodal systems. The
  toxicity-mitigated dataset is open source and is available for further
  research.
paper_authors._id: 68262cf62f7ad318b2bbb7d8
paper_authors.name: Karthik Reddy Kanjula
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2025-05-15T18:12:33.905Z
paper_authors._id: 650bd036be6db1ec2139be92
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/650bd036be6db1ec2139be92/5fp2m2x8reRkYIkVnwm1d.png
paper_authors.fullname: Karthik
paper_authors.type: user
paper_authors.user: kkr5155
paper_authors._id: 68262cf62f7ad318b2bbb7d9
paper_authors.name: Surya Guthikonda
paper_authors._id: 68262cf62f7ad318b2bbb7da
paper_authors.name: Nahid Alam
paper_authors._id: 68262cf62f7ad318b2bbb7db
paper_authors.name: Shayekh Bin Islam
paper_title: Understanding and Mitigating Toxicity in Image-Text Pretraining
    Datasets: A Case Study on LLaVA
paper_summary: Pretraining datasets are foundational to the development of multimodal
  models, yet they often have inherent biases and toxic content from the
  web-scale corpora they are sourced from. In this paper, we investigate the
  prevalence of toxicity in LLaVA image-text pretraining dataset, examining how
  harmful content manifests in different modalities. We present a comprehensive
  analysis of common toxicity categories and propose targeted mitigation
  strategies, resulting in the creation of a refined toxicity-mitigated dataset.
  This dataset removes 7,531 of toxic image-text pairs in the LLaVA pre-training
  dataset. We offer guidelines for implementing robust toxicity detection
  pipelines. Our findings underscore the need to actively identify and filter
  toxic content - such as hate speech, explicit imagery, and targeted harassment
  - to build more responsible and equitable multimodal systems. The
  toxicity-mitigated dataset is open source and is available for further
  research.
paper_ai_summary: A detailed analysis of toxicity in the LLaVA image-text pretraining dataset identifies common types of harmful content and proposes strategies to mitigate it, resulting in a refined open-source dataset.
paper_ai_keywords: LLaVA image-text pretraining dataset multimodal models toxicity toxicity-mitigated dataset hate speech explicit imagery targeted harassment robust toxicity detection pipelines