Teaching Models to Understand (but not Generate) High-risk Data
Language model developers typically filter out high-risk content -- such as
toxic or copyrighted text -- from their pre-training data to prevent models
from generating similar outputs. However, removing such data altogether limits
models' ability to recognize and appropriately respond to harmful or sensitive
content. In this paper, we introduce Selective Loss to Understand but Not
Generate (SLUNG), a pre-training paradigm through which models learn to
understand high-risk data without learning to generate it. Instead of uniformly
applying the next-token prediction loss, SLUNG selectively avoids incentivizing
the generation of high-risk tokens while ensuring they remain within the
model's context window. As the model learns to predict low-risk tokens that
follow high-risk ones, it is forced to understand the high-risk content.
Through our experiments, we show that SLUNG consistently improves models'
understanding of high-risk data (e.g., ability to recognize toxic content)
without increasing its generation (e.g., toxicity of model responses). Overall,
our SLUNG paradigm enables models to benefit from high-risk text that would
otherwise be filtered out.
681bb31051367fb0a7201e92
Ryan Wang
claimed_verified
2025-05-13T08:13:08.080Z
653ae13eb0f7bd82e0e4c08a
/avatars/469614615e437c2759f9bdf42a2cdc9a.svg
Ryan Wang
user
ryanyxw
681bb31051367fb0a7201e93
Matthew Finlayson
claimed_verified
2025-05-08T08:59:14.472Z
62698a1284ad29bad6033263
/avatars/ca117f3364e7fb16b09612eab5226c4f.svg
Matthew Finlayson
user
mattf1n
681bb31051367fb0a7201e94
Luca Soldaini
681bb31051367fb0a7201e95
Swabha Swayamdipta
claimed_verified
2025-07-31T08:16:31.286Z
688b03af9c6feb94ac299fc9
/avatars/bf94918c198fe866a7d4056ea887108f.svg
Swabha Swayamdipta
user
swabhs
681bb31051367fb0a7201e96
Robin Jia
Teaching Models to Understand (but not Generate) High-risk Data
Language model developers typically filter out high-risk content -- such as
toxic or copyrighted text -- from their pre-training data to prevent models
from generating similar outputs. However, removing such data altogether limits
models' ability to recognize and appropriately respond to harmful or sensitive
content. In this paper, we introduce Selective Loss to Understand but Not
Generate (SLUNG), a pre-training paradigm through which models learn to
understand high-risk data without learning to generate it. Instead of uniformly
applying the next-token prediction loss, SLUNG selectively avoids incentivizing
the generation of high-risk tokens while ensuring they remain within the
model's context window. As the model learns to predict low-risk tokens that
follow high-risk ones, it is forced to understand the high-risk content.
Through our experiments, we show that SLUNG consistently improves models'
understanding of high-risk data (e.g., ability to recognize toxic content)
without increasing its generation (e.g., toxicity of model responses). Overall,
our SLUNG paradigm enables models to benefit from high-risk text that would
otherwise be filtered out.
The SLUNG paradigm allows models to understand high-risk content without generating it, enhancing their ability to recognize such content while preventing its generation.
Selective Loss SLUNG pre-training paradigm next-token prediction loss high-risk tokens context window toxicity model responses