title: MetaSC: Test-Time Safety Specification Optimization for Language Models
summary: We propose a novel dynamic safety framework that optimizes language model
  (LM) safety reasoning at inference time without modifying model weights.
  Building on recent advances in self-critique methods, our approach leverages a
  meta-critique mechanism that iteratively updates safety prompts-termed
  specifications-to drive the critique and revision process adaptively. This
  test-time optimization not only improves performance against adversarial
  jailbreak requests but also in diverse general safety-related tasks, such as
  avoiding moral harm or pursuing honest responses. Our empirical evaluations
  across several language models demonstrate that dynamically optimized safety
  prompts yield significantly higher safety scores compared to fixed system
  prompts and static self-critique defenses. Code to be released at
  https://github.com/vicgalle/meta-self-critique.git .
paper_authors._id: 67ad9577b469222e0df18134
paper_authors.name: Víctor Gallego
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2025-02-13T06:47:20.731Z
paper_authors._id: 5fad8602b8423e1d80b8a965
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/5fad8602b8423e1d80b8a965/tRqTwcZmrGka8c1vFq2wX.jpeg
paper_authors.fullname: Victor Gallego
paper_authors.type: user
paper_authors.user: vicgalle
paper_title: MetaSC: Test-Time Safety Specification Optimization for Language Models
paper_summary: We propose a novel dynamic safety framework that optimizes language model
  (LM) safety reasoning at inference time without modifying model weights.
  Building on recent advances in self-critique methods, our approach leverages a
  meta-critique mechanism that iteratively updates safety prompts-termed
  specifications-to drive the critique and revision process adaptively. This
  test-time optimization not only improves performance against adversarial
  jailbreak requests but also in diverse general safety-related tasks, such as
  avoiding moral harm or pursuing honest responses. Our empirical evaluations
  across several language models demonstrate that dynamically optimized safety
  prompts yield significantly higher safety scores compared to fixed system
  prompts and static self-critique defenses. Code to be released at
  https://github.com/vicgalle/meta-self-critique.git .
paper_ai_summary: A dynamic safety framework uses meta-critique to iteratively update safety prompts, enhancing language model safety during inference without altering model weights.
paper_ai_keywords: language model self-critique methods meta-critique mechanism safety prompts specifications critique revision process adversarial jailbreak requests moral harm honest responses safety scores fixed system prompts static self-critique defenses