Fast Training Dataset Attribution via In-Context Learning
<|reference_start|>Fast Training Dataset Attribution via In-Context Learning: We investigate the use of in-context learning and prompt engineering to estimate the contributions of training data in the outputs of instruction-tuned large language models (LLMs). We propose two novel approaches: (1) a similarity-based approach that measures the difference between LLM outputs with and without provided context, and (2) a mixture distribution model approach that frames the problem of identifying contribution scores as a matrix factorization task. Our empirical comparison demonstrates that the mixture model approach is more robust to retrieval noise in in-context learning, providing a more reliable estimation of data contributions.<|reference_end|>
@article{fotouhi2024fast,
    title={Fast Training Dataset Attribution via In-Context Learning},
    author={Milad Fotouhi, Mohammad Taha Bahadori, Oluwaseyi Feyisetan, Payman
  Arabshahi, David Heckerman},
    journal={arXiv preprint arXiv:2408.11852},
    year={2024},
    archivePrefix={arXiv},
    eprint={2408.11852},
    primaryClass={cs.CL cs.AI cs.LG}
}