title: Phare: A Safety Probe for Large Language Models
summary: Ensuring the safety of large language models (LLMs) is critical for
  responsible deployment, yet existing evaluations often prioritize performance
  over identifying failure modes. We introduce Phare, a multilingual diagnostic
  framework to probe and evaluate LLM behavior across three critical dimensions:
  hallucination and reliability, social biases, and harmful content generation.
  Our evaluation of 17 state-of-the-art LLMs reveals patterns of systematic
  vulnerabilities across all safety dimensions, including sycophancy, prompt
  sensitivity, and stereotype reproduction. By highlighting these specific
  failure modes rather than simply ranking models, Phare provides researchers and
  practitioners with actionable insights to build more robust, aligned, and
  trustworthy language systems.
paper_authors._id: 682d7ae082567fffe108b31a
paper_authors.name: Pierre Le Jeune
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2025-05-21T07:16:41.247Z
paper_authors._id: 6596ca5cce76219628b8eab4
paper_authors.avatarUrl: /avatars/51cdea4e1e0e53260d403ceb7bc6de90.svg
paper_authors.fullname: Pierre Le Jeune
paper_authors.type: user
paper_authors.user: pierlj
paper_authors._id: 682d7ae082567fffe108b31b
paper_authors.name: Benoît Malézieux
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2025-05-21T07:04:02.669Z
paper_authors._id: 65bccff4c1a44b6ef1c100da
paper_authors.avatarUrl: /avatars/bf91000c78b83167958dc44c582397f0.svg
paper_authors.fullname: benoit
paper_authors.type: user
paper_authors.user: bmalezieux
paper_authors._id: 682d7ae082567fffe108b31c
paper_authors.name: Weixuan Xiao
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2025-05-21T07:04:02.669Z
paper_authors._id: 649f00fc37bfb5202be464a9
paper_authors.avatarUrl: /avatars/89f6c6a92c076099f5450c3cd2057619.svg
paper_authors.fullname: Weixuan Xiao
paper_authors.type: user
paper_authors.user: weixuan-giskard
paper_authors._id: 682d7ae082567fffe108b31d
paper_authors.name: Matteo Dora
paper_title: Phare: A Safety Probe for Large Language Models
paper_summary: Ensuring the safety of large language models (LLMs) is critical for
  responsible deployment, yet existing evaluations often prioritize performance
  over identifying failure modes. We introduce Phare, a multilingual diagnostic
  framework to probe and evaluate LLM behavior across three critical dimensions:
  hallucination and reliability, social biases, and harmful content generation.
  Our evaluation of 17 state-of-the-art LLMs reveals patterns of systematic
  vulnerabilities across all safety dimensions, including sycophancy, prompt
  sensitivity, and stereotype reproduction. By highlighting these specific
  failure modes rather than simply ranking models, Phare provides researchers and
  practitioners with actionable insights to build more robust, aligned, and
  trustworthy language systems.
paper_ai_summary: Phare evaluates large language models across safety dimensions to uncover specific failure modes, offering insights for building more robust systems.
paper_ai_keywords: large language models hallucination reliability social biases harmful content generation sycophancy prompt sensitivity stereotype reproduction