Controlling Language and Diffusion Models by Transporting Activations
The increasing capabilities of large generative models and their ever more
widespread deployment have raised concerns about their reliability, safety, and
potential misuse. To address these issues, recent works have proposed to
control model generation by steering model activations in order to effectively
induce or prevent the emergence of concepts or behaviors in the generated
output. In this paper we introduce Activation Transport (AcT), a general
framework to steer activations guided by optimal transport theory that
generalizes many previous activation-steering works. AcT is modality-agnostic
and provides fine-grained control over the model behavior with negligible
computational overhead, while minimally impacting model abilities. We
experimentally show the effectiveness and versatility of our approach by
addressing key challenges in large language models (LLMs) and text-to-image
diffusion models (T2Is). For LLMs, we show that AcT can effectively mitigate
toxicity, induce arbitrary concepts, and increase their truthfulness. In T2Is,
we show how AcT enables fine-grained style control and concept negation.
672b447c1b5f7a5e97a026e8
Pau Rodriguez
672b447c1b5f7a5e97a026e9
Arno Blaas
672b447c1b5f7a5e97a026ea
Michal Klein
672b447c1b5f7a5e97a026eb
Luca Zappella
672b447c1b5f7a5e97a026ec
Nicholas Apostoloff
672b447c1b5f7a5e97a026ed
Marco Cuturi
672b447c1b5f7a5e97a026ee
Xavier Suau
Controlling Language and Diffusion Models by Transporting Activations
The increasing capabilities of large generative models and their ever more
widespread deployment have raised concerns about their reliability, safety, and
potential misuse. To address these issues, recent works have proposed to
control model generation by steering model activations in order to effectively
induce or prevent the emergence of concepts or behaviors in the generated
output. In this paper we introduce Activation Transport (AcT), a general
framework to steer activations guided by optimal transport theory that
generalizes many previous activation-steering works. AcT is modality-agnostic
and provides fine-grained control over the model behavior with negligible
computational overhead, while minimally impacting model abilities. We
experimentally show the effectiveness and versatility of our approach by
addressing key challenges in large language models (LLMs) and text-to-image
diffusion models (T2Is). For LLMs, we show that AcT can effectively mitigate
toxicity, induce arbitrary concepts, and increase their truthfulness. In T2Is,
we show how AcT enables fine-grained style control and concept negation.
Activation Transport (AcT) is a framework that uses optimal transport theory to control model activations, effectively addressing challenges in large language models and text-to-image diffusion models.
Activation Transport AcT optimal transport theory modality-agnostic fine-grained control computational overhead toxicity truthfulness fine-grained style control concept negation