A simple way to overcome these limitations of
today’s language models is to give them the abil-
ity to use external tools such as search engines,
calculators, or calendars. However, existing ap-
proaches either rely on large amounts of human
annotations (Komeili et al., 2022; Thoppilan et al.,
2022) or limit tool use to task-speciﬁc settings only
(e.g., Gao et al., 2022; Parisi et al., 2022), hinder-
ing a more widespread adoption of tool use in LMs.
Therefore, we propose Toolformer, a model that
learns to use tools in a novel way, which fulﬁlls the
following desiderata:

• The use of tools should be learned in a
self-supervised way without requiring large
amounts of human annotations. This is impor-
Toolformer