Model Editing with Canonical Examples
We introduce model editing with canonical examples, a setting in which (1) a
single learning example is provided per desired behavior, (2) evaluation is
performed exclusively out-of-distribution, and (3) deviation from an initial
model is strictly limited. A canonical example is a simple instance of good
behavior, e.g., The capital of Mauritius is Port Louis) or bad behavior, e.g.,
An aspect of researchers is coldhearted). The evaluation set contains more
complex examples of each behavior (like a paragraph in which the capital of
Mauritius is called for.) We create three datasets and modify three more for
model editing with canonical examples, covering knowledge-intensive
improvements, social bias mitigation, and syntactic edge cases. In our
experiments on Pythia language models, we find that LoRA outperforms full
finetuning and MEMIT. We then turn to the Backpack language model architecture
because it is intended to enable targeted improvement. The Backpack defines a
large bank of sense vectors--a decomposition of the different uses of each
word--which are weighted and summed to form the output logits of the model. We
propose sense finetuning, which selects and finetunes a few (approx 10)
sense vectors for each canonical example, and find that it outperforms other
finetuning methods, e.g., 4.8% improvement vs 0.3%. Finally, we improve
GPT-J-6B by an inference-time ensemble with just the changes from sense
finetuning of a 35x smaller Backpack, in one setting outperforming editing
GPT-J itself (4.1% vs 1.0%).
65c9890fb6d39ca60c7be13e
John Hewitt
claimed_verified
2024-02-12T08:39:06.553Z
646ec1161a198f8520f3c705
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/QPCRraySl9zTnf7eG5ZJk.png
John Hewitt
user
johnhew
65c9890fb6d39ca60c7be13f
Sarah Chen
extracted_pending
2024-02-12T02:57:19.935Z
635615dca5546ae4c5cd59df
/avatars/463088bdcd6d256528e76257b7185d33.svg
Sarah Chen
user
sachen
65c9890fb6d39ca60c7be140
Lanruo Lora Xie
extracted_confirmed
2024-02-12T03:02:56.983Z
643a1ba58e214584643ae674
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/JxO8r2I_EutismZQxBS76.jpeg
Lora Xie
user
lora-x
65c9890fb6d39ca60c7be141
Edward Adams
admin_assigned
2024-02-12T17:05:57.442Z
646c312cdf2609a541c548f5
/avatars/c193eab2802a0db657b3dea09d1ba6bc.svg
Edward Adams
user
Trendoplasm
65c9890fb6d39ca60c7be142
Percy Liang
admin_assigned
2024-02-12T17:06:03.817Z
6409651b9e9f790c905b2335
/avatars/1fb8c80b60f21f65a0a027319101f236.svg
Percy Liang
user
percyliang
65c9890fb6d39ca60c7be143
Christopher D. Manning
admin_assigned
2024-02-12T17:06:25.146Z
5f555a1f8bf55658acfed19e
https://cdn-avatars.huggingface.co/v1/production/uploads/1628695407761-5f555a1f8bf55658acfed19e.jpeg
Christopher Manning
user
manning
Model Editing with Canonical Examples
We introduce model editing with canonical examples, a setting in which (1) a
single learning example is provided per desired behavior, (2) evaluation is
performed exclusively out-of-distribution, and (3) deviation from an initial
model is strictly limited. A canonical example is a simple instance of good
behavior, e.g., The capital of Mauritius is Port Louis) or bad behavior, e.g.,
An aspect of researchers is coldhearted). The evaluation set contains more
complex examples of each behavior (like a paragraph in which the capital of
Mauritius is called for.) We create three datasets and modify three more for
model editing with canonical examples, covering knowledge-intensive
improvements, social bias mitigation, and syntactic edge cases. In our
experiments on Pythia language models, we find that LoRA outperforms full
finetuning and MEMIT. We then turn to the Backpack language model architecture
because it is intended to enable targeted improvement. The Backpack defines a
large bank of sense vectors--a decomposition of the different uses of each
word--which are weighted and summed to form the output logits of the model. We
propose sense finetuning, which selects and finetunes a few (approx 10)
sense vectors for each canonical example, and find that it outperforms other
finetuning methods, e.g., 4.8% improvement vs 0.3%. Finally, we improve
GPT-J-6B by an inference-time ensemble with just the changes from sense
finetuning of a 35x smaller Backpack, in one setting outperforming editing
GPT-J itself (4.1% vs 1.0%).
Model editing using canonical examples improves performance with LoRA and sense finetuning on Pythia and Backpack language models.
LoRA full finetuning MEMIT Backpack sense vectors sense finetuning output logits inference-time ensemble