and enables one to interpret others’ feedback. The emergence of LLMs highlights the potential
of foundation models to understand human intentions, but when it comes to vague instructions or
other implications, it poses a significant challenge for agents [94; 136]. For humans, grasping the
implied meanings from a conversation comes naturally, whereas for agents, they should formalize
implied meanings into a reward function that allows them to choose the option in line with the
speaker’s preferences in unseen contexts [128]. One of the main ways for reward modeling is
inferring rewards based on feedback, which is primarily presented in the form of comparisons [218]
(possibly supplemented with reasons [219]) and unconstrained natural language [220]. Another way
involves recovering rewards from descriptions, using the action space as a bridge [128]. Jeon et al.
[221] suggests that human behavior can be mapped to a choice from an implicit set of options, which
TheRiseandPotentialofLargeLanguageModel BasedAgents