title: Watermarking Degrades Alignment in Language Models: Analysis and
    Mitigation
summary: Watermarking techniques for large language models (LLMs) can significantly
  impact output quality, yet their effects on truthfulness, safety, and
  helpfulness remain critically underexamined. This paper presents a systematic
  analysis of how two popular watermarking approaches-Gumbel and KGW-affect these
  core alignment properties across four aligned LLMs. Our experiments reveal two
  distinct degradation patterns: guard attenuation, where enhanced helpfulness
  undermines model safety, and guard amplification, where excessive caution
  reduces model helpfulness. These patterns emerge from watermark-induced shifts
  in token distribution, surfacing the fundamental tension that exists between
  alignment objectives.
    To mitigate these degradations, we propose Alignment Resampling (AR), an
  inference-time sampling method that uses an external reward model to restore
  alignment. We establish a theoretical lower bound on the improvement in
  expected reward score as the sample size is increased and empirically
  demonstrate that sampling just 2-4 watermarked generations effectively recovers
  or surpasses baseline (unwatermarked) alignment scores. To overcome the limited
  response diversity of standard Gumbel watermarking, our modified implementation
  sacrifices strict distortion-freeness while maintaining robust detectability,
  ensuring compatibility with AR. Experimental results confirm that AR
  successfully recovers baseline alignment in both watermarking approaches, while
  maintaining strong watermark detectability. This work reveals the critical
  balance between watermark strength and model alignment, providing a simple
  inference-time solution to responsibly deploy watermarked LLMs in practice.
paper_authors._id: 6842fc77a4e3571765bb4951
paper_authors.name: Apurv Verma
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-07T05:45:19.783Z
paper_authors._id: 5f9f8cc7a13e063b8b2b5bdc
paper_authors.avatarUrl: /avatars/46e4669ef1b08e449fd46bec60eb66e8.svg
paper_authors.fullname: Apurv
paper_authors.type: user
paper_authors.user: 0xe69756
paper_authors._id: 6842fc77a4e3571765bb4952
paper_authors.name: NhatHai Phan
paper_authors._id: 6842fc77a4e3571765bb4953
paper_authors.name: Shubhendu Trivedi
paper_title: Watermarking Degrades Alignment in Language Models: Analysis and
    Mitigation
paper_summary: Watermarking techniques for large language models (LLMs) can significantly
  impact output quality, yet their effects on truthfulness, safety, and
  helpfulness remain critically underexamined. This paper presents a systematic
  analysis of how two popular watermarking approaches-Gumbel and KGW-affect these
  core alignment properties across four aligned LLMs. Our experiments reveal two
  distinct degradation patterns: guard attenuation, where enhanced helpfulness
  undermines model safety, and guard amplification, where excessive caution
  reduces model helpfulness. These patterns emerge from watermark-induced shifts
  in token distribution, surfacing the fundamental tension that exists between
  alignment objectives.
    To mitigate these degradations, we propose Alignment Resampling (AR), an
  inference-time sampling method that uses an external reward model to restore
  alignment. We establish a theoretical lower bound on the improvement in
  expected reward score as the sample size is increased and empirically
  demonstrate that sampling just 2-4 watermarked generations effectively recovers
  or surpasses baseline (unwatermarked) alignment scores. To overcome the limited
  response diversity of standard Gumbel watermarking, our modified implementation
  sacrifices strict distortion-freeness while maintaining robust detectability,
  ensuring compatibility with AR. Experimental results confirm that AR
  successfully recovers baseline alignment in both watermarking approaches, while
  maintaining strong watermark detectability. This work reveals the critical
  balance between watermark strength and model alignment, providing a simple
  inference-time solution to responsibly deploy watermarked LLMs in practice.
paper_ai_summary: Watermarking techniques in large language models affect core alignment properties, and Alignment Resampling can mitigate these effects while maintaining watermark detectability.
paper_ai_keywords: Gumbel KGW Alignment Resampling token distribution alignment objectives expected reward score sample size