3.1.2 Knowledge

Due to the diversity of the real world, many NLP researchers attempt to utilize data that has a larger
scale. This data usually is unstructured and unlabeled [137; 138], yet it contains enormous knowledge
that language models could learn. In theory, language models can learn more knowledge as they have
more parameters [139], and it is possible for language models to learn and comprehend everything in
natural language. Research [140] shows that language models trained on a large-scale dataset can
encode a wide range of knowledge into their parameters and respond correctly to various types of
queries. Furthermore, the knowledge can assist LLM-based agents in making informed decisions
[222]. All of this knowledge can be roughly categorized into the following types:
TheRiseandPotentialofLargeLanguageModel BasedAgents