Data Valuation using Neural Networks for Efficient Instruction
  Fine-Tuning
Influence functions provide crucial insights into model training, but
existing methods suffer from large computational costs and limited
generalization. Particularly, recent works have proposed various metrics and
algorithms to calculate the influence of data using language models, which do
not scale well with large models and datasets. This is because of the expensive
forward and backward passes required for computation, substantial memory
requirements to store large models, and poor generalization of influence
estimates to new data. In this paper, we explore the use of small neural
networks -- which we refer to as the InfluenceNetwork -- to estimate influence
values, achieving up to 99% cost reduction. Our evaluation demonstrates that
influence values can be estimated with models just 0.0027% the size of full
language models (we use 7B and 8B versions). We apply our algorithm of
estimating influence values (called NN-CIFT: Neural Networks for effiCient
Instruction Fine-Tuning) to the downstream task of subset selection for general
instruction fine-tuning. In our study, we include four state-of-the-art
influence functions and show no compromise in performance, despite large
speedups, between NN-CIFT and the original influence functions. We provide an
in-depth hyperparameter analyses of NN-CIFT. The code for our method can be
found here: https://github.com/agarwalishika/NN-CIFT.
67b4cb6c777b7676c8b3c43d
Ishika Agarwal
extracted_confirmed
2025-02-18T18:06:42.786Z
6391e4e984afa726d66180b9
/avatars/e437e2820745b522a868b8da27d9a11f.svg
Ishika Agarwal
user
ishikaa
67b4cb6c777b7676c8b3c43e
Dilek Hakkani-Tür
extracted_confirmed
2025-04-02T17:58:22.304Z
6747519282137543033451d9
/avatars/8bd763f19d8e9f97f0f6817435b062dc.svg
Dilek Hakkani-Tur
user
dilekht
Data Valuation using Neural Networks for Efficient Instruction
  Fine-Tuning
Influence functions provide crucial insights into model training, but
existing methods suffer from large computational costs and limited
generalization. Particularly, recent works have proposed various metrics and
algorithms to calculate the influence of data using language models, which do
not scale well with large models and datasets. This is because of the expensive
forward and backward passes required for computation, substantial memory
requirements to store large models, and poor generalization of influence
estimates to new data. In this paper, we explore the use of small neural
networks -- which we refer to as the InfluenceNetwork -- to estimate influence
values, achieving up to 99% cost reduction. Our evaluation demonstrates that
influence values can be estimated with models just 0.0027% the size of full
language models (we use 7B and 8B versions). We apply our algorithm of
estimating influence values (called NN-CIFT: Neural Networks for effiCient
Instruction Fine-Tuning) to the downstream task of subset selection for general
instruction fine-tuning. In our study, we include four state-of-the-art
influence functions and show no compromise in performance, despite large
speedups, between NN-CIFT and the original influence functions. We provide an
in-depth hyperparameter analyses of NN-CIFT. The code for our method can be
found here: https://github.com/agarwalishika/NN-CIFT.
Neural Networks for efficient Instruction Fine-Tuning (NN-CIFT) significantly reduce the computational cost of estimating influence values while maintaining performance for subset selection in instruction fine-tuning.
Influence functions InfluenceNetwork NN-CIFT instruction fine-tuning subset selection state-of-the-art influence functions hyperparameter analyses