title: ReFT: Representation Finetuning for Language Models
summary: Parameter-efficient fine-tuning (PEFT) methods seek to adapt large models via
  updates to a small number of weights. However, much prior interpretability work
  has shown that representations encode rich semantic information, suggesting
  that editing representations might be a more powerful alternative. Here, we
  pursue this hypothesis by developing a family of Representation
  Finetuning (ReFT) methods. ReFT methods operate on a frozen base model and
  learn task-specific interventions on hidden representations. We define a strong
  instance of the ReFT family, Low-rank Linear Subspace ReFT (LoReFT). LoReFT is
  a drop-in replacement for existing PEFTs and learns interventions that are
  10x-50x more parameter-efficient than prior state-of-the-art PEFTs. We showcase
  LoReFT on eight commonsense reasoning tasks, four arithmetic reasoning tasks,
  Alpaca-Eval v1.0, and GLUE. In all these evaluations, LoReFT delivers the best
  balance of efficiency and performance, and almost always outperforms
  state-of-the-art PEFTs. We release a generic ReFT training library publicly at
  https://github.com/stanfordnlp/pyreft.
paper_authors._id: 660f65b29ff4d088f566343e
paper_authors.name: Zhengxuan Wu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-04-06T15:19:32.200Z
paper_authors._id: 604fa06d4f9b833fc41c75cf
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1650432433568-604fa06d4f9b833fc41c75cf.png
paper_authors.fullname: Zhengxuan Wu
paper_authors.type: user
paper_authors.user: zhengxuanzenwu
paper_authors._id: 660f65b29ff4d088f566343f
paper_authors.name: Aryaman Arora
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-04-05T10:14:26.373Z
paper_authors._id: 603803ad1b4a9bee818ab78e
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/603803ad1b4a9bee818ab78e/CLWFX4lUD0hxJsV7uiiQE.jpeg
paper_authors.fullname: Aryaman Arora
paper_authors.type: user
paper_authors.user: aryaman
paper_authors._id: 660f65b29ff4d088f5663440
paper_authors.name: Zheng Wang
paper_authors._id: 660f65b29ff4d088f5663441
paper_authors.name: Atticus Geiger
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-05-27T07:21:07.672Z
paper_authors._id: 627b2d0527dc4650b62eef42
paper_authors.avatarUrl: /avatars/e70381850f5657b54e90f5539f3d74eb.svg
paper_authors.fullname: Atticus Geiger
paper_authors.type: user
paper_authors.user: atticusg
paper_authors._id: 660f65b29ff4d088f5663442
paper_authors.name: Dan Jurafsky
paper_authors._id: 660f65b29ff4d088f5663443
paper_authors.name: Christopher D. Manning
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-04-25T07:37:50.983Z
paper_authors._id: 5f555a1f8bf55658acfed19e
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1628695407761-5f555a1f8bf55658acfed19e.jpeg
paper_authors.fullname: Christopher Manning
paper_authors.type: user
paper_authors.user: manning
paper_authors._id: 660f65b29ff4d088f5663444
paper_authors.name: Christopher Potts
paper_title: ReFT: Representation Finetuning for Language Models
paper_summary: Parameter-efficient fine-tuning (PEFT) methods seek to adapt large models via
  updates to a small number of weights. However, much prior interpretability work
  has shown that representations encode rich semantic information, suggesting
  that editing representations might be a more powerful alternative. Here, we
  pursue this hypothesis by developing a family of Representation
  Finetuning (ReFT) methods. ReFT methods operate on a frozen base model and
  learn task-specific interventions on hidden representations. We define a strong
  instance of the ReFT family, Low-rank Linear Subspace ReFT (LoReFT). LoReFT is
  a drop-in replacement for existing PEFTs and learns interventions that are
  10x-50x more parameter-efficient than prior state-of-the-art PEFTs. We showcase
  LoReFT on eight commonsense reasoning tasks, four arithmetic reasoning tasks,
  Alpaca-Eval v1.0, and GLUE. In all these evaluations, LoReFT delivers the best
  balance of efficiency and performance, and almost always outperforms
  state-of-the-art PEFTs. We release a generic ReFT training library publicly at
  https://github.com/stanfordnlp/pyreft.
paper_ai_summary: Representation Finetuning (ReFT) methods, exemplified by Low-rank Linear Subspace ReFT (LoReFT), achieve high efficiency and performance by adapting representations in frozen base models, outperforming state-of-the-art Parameter-efficient Fine-tuning (PEFT) methods.
paper_ai_keywords: Parameter-efficient fine-tuning Representation Finetuning ReFT Low-rank Linear Subspace ReFT LoReFT hidden representations commonsense reasoning arithmetic reasoning Alpaca-Eval GLUE