Unseen task generalization. Studies show that instruction-tuned LLMs exhibit zero-shot gener-
alization without the need for task-specific fine-tuning [24; 25; 105; 106; 107]. With the expansion
of model size and corpus size, LLMs gradually exhibit remarkable emergent abilities in unfamiliar
tasks [132]. Specifically, LLMs can complete new tasks they do not encounter in the training stage by
following the instructions based on their own understanding. One of the implementations is multi-task
learning, for example, FLAN [105] finetunes language models on a collection of tasks described via
instructions, and T0 [106] introduces a unified framework that converts every language problem into
a text-to-text format. Despite being purely a language model, GPT-4 [25] demonstrates remarkable
capabilities in a variety of domains and tasks, including abstraction, comprehension, vision, coding,
mathematics, medicine, law, understanding of human motives and emotions, and others [31]. It is
TheRiseandPotentialofLargeLanguageModel BasedAgents