Consecutive Model Editing with Batch alongside HooK Layers
<|reference_start|>Consecutive Model Editing with Batch alongside HooK Layers: As the typical retraining paradigm is unacceptably time- and resource-consuming, researchers are turning to model editing in order to seek an effective, consecutive, and batch-supportive way to edit the model behavior directly. Despite all these practical expectations, existing model editing methods fail to realize all of them. Furthermore, the memory demands for such succession-supportive model editing approaches tend to be prohibitive, frequently necessitating an external memory that grows incrementally over time. To cope with these challenges, we propose COMEBA-HK, a model editing method that is both consecutive and batch-supportive. COMEBA-HK is memory-friendly as it only needs a small amount of it to store several hook layers with updated weights. Experimental results demonstrate the superiority of our method over other batch-supportive model editing methods under both single-round and consecutive batch editing scenarios. Extensive analyses of COMEBA-HK have been conducted to verify the stability of our method over 1) the number of consecutive steps and 2) the number of editing instance.<|reference_end|>
@article{li2024consecutive,
    title={Consecutive Batch Model Editing with HooK Layers},
    author={Shuaiyi Li, Yang Deng, Deng Cai, Hongyuan Lu, Liang Chen, Wai Lam},
    journal={arXiv preprint arXiv:2403.05330},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.05330},
    primaryClass={cs.CL}
}