Watermarking Degrades Alignment in Language Models: Analysis and
  Mitigation
Watermarking techniques for large language models (LLMs) can significantly
impact output quality, yet their effects on truthfulness, safety, and
helpfulness remain critically underexamined. This paper presents a systematic
analysis of how two popular watermarking approaches-Gumbel and KGW-affect these
core alignment properties across four aligned LLMs. Our experiments reveal two
distinct degradation patterns: guard attenuation, where enhanced helpfulness
undermines model safety, and guard amplification, where excessive caution
reduces model helpfulness. These patterns emerge from watermark-induced shifts
in token distribution, surfacing the fundamental tension that exists between
alignment objectives.
  To mitigate these degradations, we propose Alignment Resampling (AR), an
inference-time sampling method that uses an external reward model to restore
alignment. We establish a theoretical lower bound on the improvement in
expected reward score as the sample size is increased and empirically
demonstrate that sampling just 2-4 watermarked generations effectively recovers
or surpasses baseline (unwatermarked) alignment scores. To overcome the limited
response diversity of standard Gumbel watermarking, our modified implementation
sacrifices strict distortion-freeness while maintaining robust detectability,
ensuring compatibility with AR. Experimental results confirm that AR
successfully recovers baseline alignment in both watermarking approaches, while
maintaining strong watermark detectability. This work reveals the critical
balance between watermark strength and model alignment, providing a simple
inference-time solution to responsibly deploy watermarked LLMs in practice.
6842fc77a4e3571765bb4951
Apurv Verma
claimed_verified
2025-06-07T05:45:19.783Z
5f9f8cc7a13e063b8b2b5bdc
/avatars/46e4669ef1b08e449fd46bec60eb66e8.svg
Apurv
user
0xe69756
6842fc77a4e3571765bb4952
NhatHai Phan
6842fc77a4e3571765bb4953
Shubhendu Trivedi
Watermarking Degrades Alignment in Language Models: Analysis and
  Mitigation
Watermarking techniques for large language models (LLMs) can significantly
impact output quality, yet their effects on truthfulness, safety, and
helpfulness remain critically underexamined. This paper presents a systematic
analysis of how two popular watermarking approaches-Gumbel and KGW-affect these
core alignment properties across four aligned LLMs. Our experiments reveal two
distinct degradation patterns: guard attenuation, where enhanced helpfulness
undermines model safety, and guard amplification, where excessive caution
reduces model helpfulness. These patterns emerge from watermark-induced shifts
in token distribution, surfacing the fundamental tension that exists between
alignment objectives.
  To mitigate these degradations, we propose Alignment Resampling (AR), an
inference-time sampling method that uses an external reward model to restore
alignment. We establish a theoretical lower bound on the improvement in
expected reward score as the sample size is increased and empirically
demonstrate that sampling just 2-4 watermarked generations effectively recovers
or surpasses baseline (unwatermarked) alignment scores. To overcome the limited
response diversity of standard Gumbel watermarking, our modified implementation
sacrifices strict distortion-freeness while maintaining robust detectability,
ensuring compatibility with AR. Experimental results confirm that AR
successfully recovers baseline alignment in both watermarking approaches, while
maintaining strong watermark detectability. This work reveals the critical
balance between watermark strength and model alignment, providing a simple
inference-time solution to responsibly deploy watermarked LLMs in practice.
Watermarking techniques in large language models affect core alignment properties, and Alignment Resampling can mitigate these effects while maintaining watermark detectability.
Gumbel KGW Alignment Resampling token distribution alignment objectives expected reward score sample size