title: MULTI-STAGE INFLUENCE FUNCTION
abstract: Multi-stage training and knowledge transfer from a large-scale pretrain task to various fine-tune end tasks have revolutionized natural language processing (NLP) and computer vision (CV), with state-of-the-art performances constantly being improved. In this paper, we develop a multi-stage influence function score to track predictions from a finetune model all the way back to the pretrain data. With this score, we can identify the pretrain examples in the pretrain task that contribute most to a prediction in the fine-tune task. The proposed multi-stage influence function generalizes the original influence function for a single model in Koh et al 2017, thereby enabling influence computation through both pretrain and fine-tune models. We test our proposed method in various experiments to show its effectiveness and potential applications.
authors: Hongge Chen Si Si Yang Li Ciprian Chelba Sanjiv Kumar Duane Boning Cho-Jui Hsieh
hIndex: Cho-Jui Hsieh 58