title: Multi-property Steering of Large Language Models with Dynamic Activation
    Composition
summary: Activation steering methods were shown to be effective in conditioning
  language model generation by additively intervening over models' intermediate
  representations. However, the evaluation of these techniques has so far been
  limited to single conditioning properties and synthetic settings. In this work,
  we conduct a comprehensive evaluation of various activation steering
  strategies, highlighting the property-dependent nature of optimal parameters to
  ensure a robust effect throughout generation. To address this issue, we propose
  Dynamic Activation Composition, an information-theoretic approach to modulate
  the steering intensity of one or more properties throughout generation. Our
  experiments on multi-property steering show that our method successfully
  maintains high conditioning while minimizing the impact of conditioning on
  generation fluency.
paper_authors._id: 667bd639768e6d3e880609a9
paper_authors.name: Daniel Scalena
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-26T13:24:58.517Z
paper_authors._id: 633e9ae8630ed415eb38073c
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/633e9ae8630ed415eb38073c/FtSb_ybm08qBR1liISKQ0.jpeg
paper_authors.fullname: Daniel Scalena
paper_authors.type: user
paper_authors.user: DanielSc4
paper_authors._id: 667bd639768e6d3e880609aa
paper_authors.name: Gabriele Sarti
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2024-06-26T09:05:13.361Z
paper_authors._id: 5e7749883d77a72421292d07
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/5e7749883d77a72421292d07/M4AmBReZk_otxCIG3o0bL.jpeg
paper_authors.fullname: Gabriele Sarti
paper_authors.type: user
paper_authors.user: gsarti
paper_authors._id: 667bd639768e6d3e880609ab
paper_authors.name: Malvina Nissim
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2024-06-26T08:50:03.677Z
paper_authors._id: 6246f49c0569474a9623e05a
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1648817396108-6246f49c0569474a9623e05a.jpeg
paper_authors.fullname: Malvina Nissim
paper_authors.type: user
paper_authors.user: mnissim
paper_title: Multi-property Steering of Large Language Models with Dynamic Activation
    Composition
paper_summary: Activation steering methods were shown to be effective in conditioning
  language model generation by additively intervening over models' intermediate
  representations. However, the evaluation of these techniques has so far been
  limited to single conditioning properties and synthetic settings. In this work,
  we conduct a comprehensive evaluation of various activation steering
  strategies, highlighting the property-dependent nature of optimal parameters to
  ensure a robust effect throughout generation. To address this issue, we propose
  Dynamic Activation Composition, an information-theoretic approach to modulate
  the steering intensity of one or more properties throughout generation. Our
  experiments on multi-property steering show that our method successfully
  maintains high conditioning while minimizing the impact of conditioning on
  generation fluency.
paper_ai_summary: A comprehensive evaluation of activation steering strategies in language models reveals property-specific optimal parameters, and Dynamic Activation Composition is proposed to maintain conditioning without affecting fluency.
paper_ai_keywords: activation steering language model intermediate representations Dynamic Activation Composition information-theoretic approach multi-property steering generation fluency