title: Personalized Safety in LLMs: A Benchmark and A Planning-Based Agent
    Approach
summary: Large language models (LLMs) typically generate identical or similar
  responses for all users given the same prompt, posing serious safety risks in
  high-stakes applications where user vulnerabilities differ widely. Existing
  safety evaluations primarily rely on context-independent metrics - such as
  factuality, bias, or toxicity - overlooking the fact that the same response may
  carry divergent risks depending on the user's background or condition. We
  introduce personalized safety to fill this gap and present PENGUIN - a
  benchmark comprising 14,000 scenarios across seven sensitive domains with both
  context-rich and context-free variants. Evaluating six leading LLMs, we
  demonstrate that personalized user information significantly improves safety
  scores by 43.2%, confirming the effectiveness of personalization in safety
  alignment. However, not all context attributes contribute equally to safety
  enhancement. To address this, we develop RAISE - a training-free, two-stage
  agent framework that strategically acquires user-specific background. RAISE
  improves safety scores by up to 31.6% over six vanilla LLMs, while maintaining
  a low interaction cost of just 2.7 user queries on average. Our findings
  highlight the importance of selective information gathering in safety-critical
  domains and offer a practical solution for personalizing LLM responses without
  model retraining. This work establishes a foundation for safety research that
  adapts to individual user contexts rather than assuming a universal harm
  standard.
paper_authors._id: 6835fd8e869613d0fd6dc2fc
paper_authors.name: Yuchen Wu
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2025-05-27T17:59:44.169Z
paper_authors._id: 6543463f91a22f5a1189181a
paper_authors.avatarUrl: /avatars/6010ef307918e57e44adf2f60080fe04.svg
paper_authors.fullname: Yuchen Wu
paper_authors.type: user
paper_authors.user: wick1d
paper_authors._id: 6835fd8e869613d0fd6dc2fd
paper_authors.name: Edward Sun
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-29T20:18:48.711Z
paper_authors._id: 65822da0117b524ef3e8dbcc
paper_authors.avatarUrl: /avatars/8d3b0eed5144ff452ca2a49a0f67d7c4.svg
paper_authors.fullname: Edward Sun
paper_authors.type: user
paper_authors.user: EdwardoSunny
paper_authors._id: 6835fd8e869613d0fd6dc2fe
paper_authors.name: Kaijie Zhu
paper_authors._id: 6835fd8e869613d0fd6dc2ff
paper_authors.name: Jianxun Lian
paper_authors._id: 6835fd8e869613d0fd6dc300
paper_authors.name: Jose Hernandez-Orallo
paper_authors._id: 6835fd8e869613d0fd6dc301
paper_authors.name: Aylin Caliskan
paper_authors._id: 6835fd8e869613d0fd6dc302
paper_authors.name: Jindong Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-02T07:49:05.005Z
paper_authors._id: 6204cc0d522e40b4a18d86e2
paper_authors.avatarUrl: /avatars/18daf2de5671e711dc745388dd60569d.svg
paper_authors.fullname: Jindong Wang
paper_authors.type: user
paper_authors.user: jindongwang
paper_title: Personalized Safety in LLMs: A Benchmark and A Planning-Based Agent
    Approach
paper_summary: Large language models (LLMs) typically generate identical or similar
  responses for all users given the same prompt, posing serious safety risks in
  high-stakes applications where user vulnerabilities differ widely. Existing
  safety evaluations primarily rely on context-independent metrics - such as
  factuality, bias, or toxicity - overlooking the fact that the same response may
  carry divergent risks depending on the user's background or condition. We
  introduce personalized safety to fill this gap and present PENGUIN - a
  benchmark comprising 14,000 scenarios across seven sensitive domains with both
  context-rich and context-free variants. Evaluating six leading LLMs, we
  demonstrate that personalized user information significantly improves safety
  scores by 43.2%, confirming the effectiveness of personalization in safety
  alignment. However, not all context attributes contribute equally to safety
  enhancement. To address this, we develop RAISE - a training-free, two-stage
  agent framework that strategically acquires user-specific background. RAISE
  improves safety scores by up to 31.6% over six vanilla LLMs, while maintaining
  a low interaction cost of just 2.7 user queries on average. Our findings
  highlight the importance of selective information gathering in safety-critical
  domains and offer a practical solution for personalizing LLM responses without
  model retraining. This work establishes a foundation for safety research that
  adapts to individual user contexts rather than assuming a universal harm
  standard.
paper_ai_summary: Introducing personalized safety for LLMs through PENGUIN and RAISE frameworks Enhances safety scores by leveraging user-specific information without retraining models.
paper_ai_keywords: large language models (LLMs) personalized safety PENGUIN context-rich context-free safety alignment RAISE user-specific background safety scores selective information gathering