ReFT: Representation Finetuning for Language Models
<|reference_start|>ReFT: Representation Finetuning for Language Models: Parameter-efficient finetuning (PEFT) methods seek to adapt large neural models via updates to a small number of weights. However, much prior interpretability work has shown that representations encode rich semantic information, suggesting that editing representations might be a more powerful alternative. We pursue this hypothesis by developing a family of Representation Finetuning (ReFT) methods. ReFT methods operate on a frozen base model and learn task-specific interventions on hidden representations. We define a strong instance of the ReFT family, Low-rank Linear Subspace ReFT (LoReFT), and we identify an ablation of this method that trades some performance for increased efficiency. Both are drop-in replacements for existing PEFTs and learn interventions that are 15x--65x more parameter-efficient than LoRA. We showcase LoReFT on eight commonsense reasoning tasks, four arithmetic reasoning tasks, instruction-tuning, and GLUE. In all these evaluations, our ReFTs deliver the best balance of efficiency and performance, and almost always outperform state-of-the-art PEFTs. We release a generic ReFT training library publicly at https://github.com/stanfordnlp/pyreft.<|reference_end|>
@article{wu2024reft:,
    title={ReFT: Representation Finetuning for Language Models},
    author={Zhengxuan Wu and Aryaman Arora and Zheng Wang and Atticus Geiger and
  Dan Jurafsky and Christopher D. Manning and Christopher Potts},
    journal={arXiv preprint arXiv:2404.03592},
    year={2024},
    archivePrefix={arXiv},
    eprint={2404.03592},
    primaryClass={cs.CL cs.AI cs.LG}
}