21

3.3 Training Models for Improved Tool Learning

basic skills. With a reward function that quantiﬁes the performance of the agent in achieving the task goal, RL
has been successfully used in various tool learning scenarios, such as robotic grasping (Levine et al., 2018)
and multi-agent autocurricula (Baker et al., 2020). By optimizing the loss function, the agent learns to reﬂect
on the current state of the environment, select the appropriate tool, and perform the right actions that lead to
the highest expected reward. In the following, we introduce two sources of feedback: environment feedback
and human feedback, which can be considered sources of reward signals in the context of tool learning. These
two feedbacks are complementary and can be combined with each other.
Tool Learning with Foundation Models