Locate-then-edit for Multi-hop Factual Recall under Knowledge Editing
<|reference_start|>Locate-then-edit for Multi-hop Factual Recall under Knowledge Editing: The locate-then-edit paradigm has shown significant promise for knowledge editing (KE) in Large Language Models (LLMs). While previous methods perform well on single-hop fact recall tasks, they consistently struggle with multi-hop factual recall tasks involving newly edited knowledge. In this paper, leveraging tools in mechanistic interpretability, we first identify that in multi-hop tasks, LLMs tend to retrieve implicit subject knowledge from deeper MLP layers, unlike single-hop tasks, which rely on earlier layers. This distinction explains the poor performance of current methods in multi-hop queries, as they primarily focus on editing shallow layers, leaving deeper layers unchanged. To address this, we propose IFMET, a novel locate-then-edit KE approach designed to edit both shallow and deep MLP layers. IFMET employs multi-hop editing prompts and supplementary sets to locate and modify knowledge across different reasoning stages. Experimental results demonstrate that IFMET significantly improves performance on multi-hop factual recall tasks, effectively overcoming the limitations of previous locate-then-edit methods.<|reference_end|>
@article{zhang2024locate-then-edit,
    title={Locate-then-edit for Multi-hop Factual Recall under Knowledge Editing},
    author={Zhuoran Zhang, Yongxiang Li, Zijian Kan, Keyuan Cheng, Lijie Hu, and
  Di Wang},
    journal={arXiv preprint arXiv:2410.06331},
    year={2024},
    archivePrefix={arXiv},
    eprint={2410.06331},
    primaryClass={cs.CL cs.AI cs.LG}
}