title: RealHarm: A Collection of Real-World Language Model Application Failures
summary: Language model deployments in consumer-facing applications introduce numerous
  risks. While existing research on harms and hazards of such applications
  follows top-down approaches derived from regulatory frameworks and theoretical
  analyses, empirical evidence of real-world failure modes remains underexplored.
  In this work, we introduce RealHarm, a dataset of annotated problematic
  interactions with AI agents built from a systematic review of publicly reported
  incidents. Analyzing harms, causes, and hazards specifically from the
  deployer's perspective, we find that reputational damage constitutes the
  predominant organizational harm, while misinformation emerges as the most
  common hazard category. We empirically evaluate state-of-the-art guardrails and
  content moderation systems to probe whether such systems would have prevented
  the incidents, revealing a significant gap in the protection of AI
  applications.
paper_authors._id: 67ff741e01428cf3b865ceb5
paper_authors.name: Pierre Le Jeune
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-16T09:15:33.830Z
paper_authors._id: 6596ca5cce76219628b8eab4
paper_authors.avatarUrl: /avatars/51cdea4e1e0e53260d403ceb7bc6de90.svg
paper_authors.fullname: Pierre Le Jeune
paper_authors.type: user
paper_authors.user: pierlj
paper_authors._id: 67ff741e01428cf3b865ceb6
paper_authors.name: Jiaen Liu
paper_authors._id: 67ff741e01428cf3b865ceb7
paper_authors.name: Luca Rossi
paper_authors._id: 67ff741e01428cf3b865ceb8
paper_authors.name: Matteo Dora
paper_title: RealHarm: A Collection of Real-World Language Model Application Failures
paper_summary: Language model deployments in consumer-facing applications introduce numerous
  risks. While existing research on harms and hazards of such applications
  follows top-down approaches derived from regulatory frameworks and theoretical
  analyses, empirical evidence of real-world failure modes remains underexplored.
  In this work, we introduce RealHarm, a dataset of annotated problematic
  interactions with AI agents built from a systematic review of publicly reported
  incidents. Analyzing harms, causes, and hazards specifically from the
  deployer's perspective, we find that reputational damage constitutes the
  predominant organizational harm, while misinformation emerges as the most
  common hazard category. We empirically evaluate state-of-the-art guardrails and
  content moderation systems to probe whether such systems would have prevented
  the incidents, revealing a significant gap in the protection of AI
  applications.
paper_ai_summary: RealHarm, a dataset of real-world AI failures, reveals reputational damage and misinformation as primary risks in language model deployments, highlighting vulnerabilities in existing guardrails and content moderation systems.
paper_ai_keywords: Language model deployments AI agents RealHarm annotated problematic interactions systematic review empirical evidence organizational harm reputational damage hazard categories misinformation state-of-the-art guardrails content moderation systems