Steering Out-of-Distribution Generalization with Concept Ablation
  Fine-Tuning
Fine-tuning large language models (LLMs) can lead to unintended
out-of-distribution generalization. Standard approaches to this problem rely on
modifying training data, for example by adding data that better specify the
intended generalization. However, this is not always practical. We introduce
Concept Ablation Fine-Tuning (CAFT), a technique that leverages
interpretability tools to control how LLMs generalize from fine-tuning, without
needing to modify the training data or otherwise use data from the target
distribution. Given a set of directions in an LLM's latent space corresponding
to undesired concepts, CAFT works by ablating these concepts with linear
projections during fine-tuning, steering the model away from unintended
generalizations. We successfully apply CAFT to three fine-tuning tasks,
including emergent misalignment, a phenomenon where LLMs fine-tuned on a narrow
task generalize to give egregiously misaligned responses to general questions.
Without any changes to the fine-tuning data, CAFT reduces misaligned responses
by 10x without degrading performance on the training distribution. Overall,
CAFT represents a novel approach for steering LLM generalization without
modifying training data.
68806b4b00f4b5a05f2fbe92
Helena Casademunt
admin_assigned
2025-07-23T13:31:48.775Z
6665825fb51b58644052c0ed
/avatars/e35b62e9bbefeafb6beae3755cced70a.svg
Helena Casademunt
user
hcasademunt
68806b4b00f4b5a05f2fbe93
Caden Juang
admin_assigned
2025-07-23T13:31:54.404Z
62ebf45a341b2cd73454e796
https://cdn-avatars.huggingface.co/v1/production/uploads/62ebf45a341b2cd73454e796/kmJkeT7L3-c5XAF5suS-p.png
Caden Juang
user
kh4dien
68806b4b00f4b5a05f2fbe94
Adam Karvonen
admin_assigned
2025-07-23T13:32:00.258Z
648b69f2e9baeaafda9cf837
/avatars/cb63703500ab6878b6fd9c3a6c465527.svg
Adam Karvonen
user
adamkarvonen
68806b4b00f4b5a05f2fbe95
Samuel Marks
68806b4b00f4b5a05f2fbe96
Senthooran Rajamanoharan
68806b4b00f4b5a05f2fbe97
Neel Nanda
Steering Out-of-Distribution Generalization with Concept Ablation
  Fine-Tuning
Fine-tuning large language models (LLMs) can lead to unintended
out-of-distribution generalization. Standard approaches to this problem rely on
modifying training data, for example by adding data that better specify the
intended generalization. However, this is not always practical. We introduce
Concept Ablation Fine-Tuning (CAFT), a technique that leverages
interpretability tools to control how LLMs generalize from fine-tuning, without
needing to modify the training data or otherwise use data from the target
distribution. Given a set of directions in an LLM's latent space corresponding
to undesired concepts, CAFT works by ablating these concepts with linear
projections during fine-tuning, steering the model away from unintended
generalizations. We successfully apply CAFT to three fine-tuning tasks,
including emergent misalignment, a phenomenon where LLMs fine-tuned on a narrow
task generalize to give egregiously misaligned responses to general questions.
Without any changes to the fine-tuning data, CAFT reduces misaligned responses
by 10x without degrading performance on the training distribution. Overall,
CAFT represents a novel approach for steering LLM generalization without
modifying training data.
Concept Ablation Fine-Tuning (CAFT) uses interpretability tools to control LLM generalization by ablating undesired concepts in latent space, reducing misaligned responses without altering training data.
Concept Ablation Fine-Tuning CAFT latent space emergent misalignment LLM generalization