title: Teaching Models to Understand (but not Generate) High-risk Data
summary: Language model developers typically filter out high-risk content -- such as
  toxic or copyrighted text -- from their pre-training data to prevent models
  from generating similar outputs. However, removing such data altogether limits
  models' ability to recognize and appropriately respond to harmful or sensitive
  content. In this paper, we introduce Selective Loss to Understand but Not
  Generate (SLUNG), a pre-training paradigm through which models learn to
  understand high-risk data without learning to generate it. Instead of uniformly
  applying the next-token prediction loss, SLUNG selectively avoids incentivizing
  the generation of high-risk tokens while ensuring they remain within the
  model's context window. As the model learns to predict low-risk tokens that
  follow high-risk ones, it is forced to understand the high-risk content.
  Through our experiments, we show that SLUNG consistently improves models'
  understanding of high-risk data (e.g., ability to recognize toxic content)
  without increasing its generation (e.g., toxicity of model responses). Overall,
  our SLUNG paradigm enables models to benefit from high-risk text that would
  otherwise be filtered out.
paper_authors._id: 681bb31051367fb0a7201e92
paper_authors.name: Ryan Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-13T08:13:08.080Z
paper_authors._id: 653ae13eb0f7bd82e0e4c08a
paper_authors.avatarUrl: /avatars/469614615e437c2759f9bdf42a2cdc9a.svg
paper_authors.fullname: Ryan Wang
paper_authors.type: user
paper_authors.user: ryanyxw
paper_authors._id: 681bb31051367fb0a7201e93
paper_authors.name: Matthew Finlayson
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-08T08:59:14.472Z
paper_authors._id: 62698a1284ad29bad6033263
paper_authors.avatarUrl: /avatars/ca117f3364e7fb16b09612eab5226c4f.svg
paper_authors.fullname: Matthew Finlayson
paper_authors.type: user
paper_authors.user: mattf1n
paper_authors._id: 681bb31051367fb0a7201e94
paper_authors.name: Luca Soldaini
paper_authors._id: 681bb31051367fb0a7201e95
paper_authors.name: Swabha Swayamdipta
paper_authors._id: 681bb31051367fb0a7201e96
paper_authors.name: Robin Jia
paper_title: Teaching Models to Understand (but not Generate) High-risk Data
paper_summary: Language model developers typically filter out high-risk content -- such as
  toxic or copyrighted text -- from their pre-training data to prevent models
  from generating similar outputs. However, removing such data altogether limits
  models' ability to recognize and appropriately respond to harmful or sensitive
  content. In this paper, we introduce Selective Loss to Understand but Not
  Generate (SLUNG), a pre-training paradigm through which models learn to
  understand high-risk data without learning to generate it. Instead of uniformly
  applying the next-token prediction loss, SLUNG selectively avoids incentivizing
  the generation of high-risk tokens while ensuring they remain within the
  model's context window. As the model learns to predict low-risk tokens that
  follow high-risk ones, it is forced to understand the high-risk content.
  Through our experiments, we show that SLUNG consistently improves models'
  understanding of high-risk data (e.g., ability to recognize toxic content)
  without increasing its generation (e.g., toxicity of model responses). Overall,
  our SLUNG paradigm enables models to benefit from high-risk text that would
  otherwise be filtered out.
paper_ai_summary: The SLUNG paradigm allows models to understand high-risk content without generating it, enhancing their ability to recognize such content while preventing its generation.
paper_ai_keywords: Selective Loss SLUNG pre-training paradigm next-token prediction loss high-risk tokens context window toxicity model responses