Effective In-Context Example Selection through Data Compression
<|reference_start|>Effective In-Context Example Selection through Data Compression: In-context learning has been extensively validated in large language models. However, the mechanism and selection strategy for in-context example selection, which is a crucial ingredient in this approach, lacks systematic and in-depth research. In this paper, we propose a data compression approach to the selection of in-context examples. We introduce a two-stage method that can effectively choose relevant examples and retain sufficient information about the training dataset within the in-context examples. Our method shows a significant improvement of an average of 5.90% across five different real-world datasets using four language models.<|reference_end|>
@article{sun2024effective,
    title={Effective In-Context Example Selection through Data Compression},
    author={Zhongxiang Sun, Kepu Zhang, Haoyu Wang, Xiao Zhang, Jun Xu},
    journal={arXiv preprint arXiv:2405.11465},
    year={2024},
    archivePrefix={arXiv},
    eprint={2405.11465},
    primaryClass={cs.CL}
}