In-context learning. Numerous studies indicate that LLMs can perform a variety of complex
tasks through in-context learning (ICL), which refers to the models’ ability to learn from a few
examples in the context [195]. Few-shot in-context learning enhances the predictive performance
of language models by concatenating the original input with several complete examples as prompts
to enrich the context [41]. The key idea of ICL is learning from analogy, which is similar to the
learning process of humans [266]. Furthermore, since the prompts are written in natural language,
the interaction is interpretable and changeable, making it easier to incorporate human knowledge into
LLMs [95; 267]. Unlike the supervised learning process, ICL doesn’t involve fine-tuning or parameter
updates, which could greatly reduce the computation costs for adapting the models to new tasks.
Beyond text, researchers also explore the potential ICL capabilities in different multimodal tasks
TheRiseandPotentialofLargeLanguageModel BasedAgents