title: Model Editing with Canonical Examples
summary: We introduce model editing with canonical examples, a setting in which (1) a
  single learning example is provided per desired behavior, (2) evaluation is
  performed exclusively out-of-distribution, and (3) deviation from an initial
  model is strictly limited. A canonical example is a simple instance of good
  behavior, e.g., The capital of Mauritius is Port Louis) or bad behavior, e.g.,
  An aspect of researchers is coldhearted). The evaluation set contains more
  complex examples of each behavior (like a paragraph in which the capital of
  Mauritius is called for.) We create three datasets and modify three more for
  model editing with canonical examples, covering knowledge-intensive
  improvements, social bias mitigation, and syntactic edge cases. In our
  experiments on Pythia language models, we find that LoRA outperforms full
  finetuning and MEMIT. We then turn to the Backpack language model architecture
  because it is intended to enable targeted improvement. The Backpack defines a
  large bank of sense vectors--a decomposition of the different uses of each
  word--which are weighted and summed to form the output logits of the model. We
  propose sense finetuning, which selects and finetunes a few (approx 10)
  sense vectors for each canonical example, and find that it outperforms other
  finetuning methods, e.g., 4.8% improvement vs 0.3%. Finally, we improve
  GPT-J-6B by an inference-time ensemble with just the changes from sense
  finetuning of a 35x smaller Backpack, in one setting outperforming editing
  GPT-J itself (4.1% vs 1.0%).
paper_authors._id: 65c9890fb6d39ca60c7be13e
paper_authors.name: John Hewitt
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-02-12T08:39:06.553Z
paper_authors._id: 646ec1161a198f8520f3c705
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/QPCRraySl9zTnf7eG5ZJk.png
paper_authors.fullname: John Hewitt
paper_authors.type: user
paper_authors.user: johnhew
paper_authors._id: 65c9890fb6d39ca60c7be13f
paper_authors.name: Sarah Chen
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2024-02-12T02:57:19.935Z
paper_authors._id: 635615dca5546ae4c5cd59df
paper_authors.avatarUrl: /avatars/463088bdcd6d256528e76257b7185d33.svg
paper_authors.fullname: Sarah Chen
paper_authors.type: user
paper_authors.user: sachen
paper_authors._id: 65c9890fb6d39ca60c7be140
paper_authors.name: Lanruo Lora Xie
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2024-02-12T03:02:56.983Z
paper_authors._id: 643a1ba58e214584643ae674
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/JxO8r2I_EutismZQxBS76.jpeg
paper_authors.fullname: Lora Xie
paper_authors.type: user
paper_authors.user: lora-x
paper_authors._id: 65c9890fb6d39ca60c7be141
paper_authors.name: Edward Adams
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-12T17:05:57.442Z
paper_authors._id: 646c312cdf2609a541c548f5
paper_authors.avatarUrl: /avatars/c193eab2802a0db657b3dea09d1ba6bc.svg
paper_authors.fullname: Edward Adams
paper_authors.type: user
paper_authors.user: Trendoplasm
paper_authors._id: 65c9890fb6d39ca60c7be142
paper_authors.name: Percy Liang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-12T17:06:03.817Z
paper_authors._id: 6409651b9e9f790c905b2335
paper_authors.avatarUrl: /avatars/1fb8c80b60f21f65a0a027319101f236.svg
paper_authors.fullname: Percy Liang
paper_authors.type: user
paper_authors.user: percyliang
paper_authors._id: 65c9890fb6d39ca60c7be143
paper_authors.name: Christopher D. Manning
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-12T17:06:25.146Z
paper_authors._id: 5f555a1f8bf55658acfed19e
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1628695407761-5f555a1f8bf55658acfed19e.jpeg
paper_authors.fullname: Christopher Manning
paper_authors.type: user
paper_authors.user: manning
paper_title: Model Editing with Canonical Examples
paper_summary: We introduce model editing with canonical examples, a setting in which (1) a
  single learning example is provided per desired behavior, (2) evaluation is
  performed exclusively out-of-distribution, and (3) deviation from an initial
  model is strictly limited. A canonical example is a simple instance of good
  behavior, e.g., The capital of Mauritius is Port Louis) or bad behavior, e.g.,
  An aspect of researchers is coldhearted). The evaluation set contains more
  complex examples of each behavior (like a paragraph in which the capital of
  Mauritius is called for.) We create three datasets and modify three more for
  model editing with canonical examples, covering knowledge-intensive
  improvements, social bias mitigation, and syntactic edge cases. In our
  experiments on Pythia language models, we find that LoRA outperforms full
  finetuning and MEMIT. We then turn to the Backpack language model architecture
  because it is intended to enable targeted improvement. The Backpack defines a
  large bank of sense vectors--a decomposition of the different uses of each
  word--which are weighted and summed to form the output logits of the model. We
  propose sense finetuning, which selects and finetunes a few (approx 10)
  sense vectors for each canonical example, and find that it outperforms other
  finetuning methods, e.g., 4.8% improvement vs 0.3%. Finally, we improve
  GPT-J-6B by an inference-time ensemble with just the changes from sense
  finetuning of a 35x smaller Backpack, in one setting outperforming editing
  GPT-J itself (4.1% vs 1.0%).
paper_ai_keywords: natural model editing canonical examples out-of-distribution evaluation LoRA MEMIT Backpack language model sense vectors sense finetuning inference-time ensemble GPT-J-6B