title: Contextual Integrity in LLMs via Reasoning and Reinforcement Learning
summary: As the era of autonomous agents making decisions on behalf of users unfolds,
  ensuring contextual integrity (CI) -- what is the appropriate information to
  share while carrying out a certain task -- becomes a central question to the
  field. We posit that CI demands a form of reasoning where the agent needs to
  reason about the context in which it is operating. To test this, we first
  prompt LLMs to reason explicitly about CI when deciding what information to
  disclose. We then extend this approach by developing a reinforcement learning
  (RL) framework that further instills in models the reasoning necessary to
  achieve CI. Using a synthetic, automatically created, dataset of only sim700
  examples but with diverse contexts and information disclosure norms, we show
  that our method substantially reduces inappropriate information disclosure
  while maintaining task performance across multiple model sizes and families.
  Importantly, improvements transfer from this synthetic dataset to established
  CI benchmarks such as PrivacyLens that has human annotations and evaluates
  privacy leakage of AI assistants in actions and tool calls.
paper_authors._id: 68425054feb46a093178003f
paper_authors.name: Guangchen Lan
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2025-06-06T02:20:41.949Z
paper_authors._id: 64ff4b1a0e8369f6a8c47c7e
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64ff4b1a0e8369f6a8c47c7e/X8ocOagj89gexSa253q_8.png
paper_authors.fullname: Eric Lan
paper_authors.type: user
paper_authors.user: Eric-Lan
paper_authors._id: 68425054feb46a0931780040
paper_authors.name: Huseyin A. Inan
paper_authors._id: 68425054feb46a0931780041
paper_authors.name: Sahar Abdelnabi
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2025-06-06T10:29:16.120Z
paper_authors._id: 65e88cdd95a27dfbf6b4e63b
paper_authors.avatarUrl: /avatars/3d2d270398f0824b392f99e158e94f26.svg
paper_authors.fullname: Sahar Abdelnabi
paper_authors.type: user
paper_authors.user: sahar-abdelnabi
paper_authors._id: 68425054feb46a0931780042
paper_authors.name: Janardhan Kulkarni
paper_authors._id: 68425054feb46a0931780043
paper_authors.name: Lukas Wutschitz
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2025-06-06T09:45:54.243Z
paper_authors._id: 6380a37a5c62156ce7dff8b9
paper_authors.avatarUrl: /avatars/fbe5a20869cb55ec43759c1b5f9c4135.svg
paper_authors.fullname: Lukas Wutschitz
paper_authors.type: user
paper_authors.user: wulu
paper_authors._id: 68425054feb46a0931780044
paper_authors.name: Reza Shokri
paper_authors._id: 68425054feb46a0931780045
paper_authors.name: Christopher G. Brinton
paper_authors._id: 68425054feb46a0931780046
paper_authors.name: Robert Sim
paper_title: Contextual Integrity in LLMs via Reasoning and Reinforcement Learning
paper_summary: As the era of autonomous agents making decisions on behalf of users unfolds,
  ensuring contextual integrity (CI) -- what is the appropriate information to
  share while carrying out a certain task -- becomes a central question to the
  field. We posit that CI demands a form of reasoning where the agent needs to
  reason about the context in which it is operating. To test this, we first
  prompt LLMs to reason explicitly about CI when deciding what information to
  disclose. We then extend this approach by developing a reinforcement learning
  (RL) framework that further instills in models the reasoning necessary to
  achieve CI. Using a synthetic, automatically created, dataset of only sim700
  examples but with diverse contexts and information disclosure norms, we show
  that our method substantially reduces inappropriate information disclosure
  while maintaining task performance across multiple model sizes and families.
  Importantly, improvements transfer from this synthetic dataset to established
  CI benchmarks such as PrivacyLens that has human annotations and evaluates
  privacy leakage of AI assistants in actions and tool calls.
paper_ai_summary: A reinforcement learning framework for LLMs enhances contextual integrity by reducing inappropriate information disclosure and maintaining task performance across various benchmarks.
paper_ai_keywords: LLMs reinforcement learning contextual integrity information disclosure synthetic dataset PrivacyLens privacy leakage AI assistants