"In-Context Learning" or: How I learned to stop worrying and love "Applied Information Retrieval"
<|reference_start|>"In-Context Learning" or: How I learned to stop worrying and love "Applied Information Retrieval": With the increasing ability of large language models (LLMs), in-context learning (ICL) has evolved as a new paradigm for natural language processing (NLP), where instead of fine-tuning the parameters of an LLM specific to a downstream task with labeled examples, a small number of such examples is appended to a prompt instruction for controlling the decoder's generation process. ICL, thus, is conceptually similar to a non-parametric approach, such as $k$-NN, where the prediction for each instance essentially depends on the local topology, i.e., on a localised set of similar instances and their labels (called few-shot examples). This suggests that a test instance in ICL is analogous to a query in IR, and similar examples in ICL retrieved from a training set relate to a set of documents retrieved from a collection in IR. While standard unsupervised ranking models can be used to retrieve these few-shot examples from a training set, the effectiveness of the examples can potentially be improved by re-defining the notion of relevance specific to its utility for the downstream task, i.e., considering an example to be relevant if including it in the prompt instruction leads to a correct prediction. With this task-specific notion of relevance, it is possible to train a supervised ranking model (e.g., a bi-encoder or cross-encoder), which potentially learns to optimally select the few-shot examples. We believe that the recent advances in neural rankers can potentially find a use case for this task of optimally choosing examples for more effective downstream ICL predictions.<|reference_end|>
@article{parry2024"in-context,
    title={"In-Context Learning" or: How I learned to stop worrying and love
  "Applied Information Retrieval"},
    author={Andrew Parry, Debasis Ganguly, Manish Chandra},
    journal={arXiv preprint arXiv:2405.01116},
    year={2024},
    doi={10.1145/3626772.3657842},
    archivePrefix={arXiv},
    eprint={2405.01116},
    primaryClass={cs.IR}
}