4 Experiments

We investigate whether our approach enables a
model to use tools without any further supervision
and to decide for itself when and how to call which
of the available tools. To test this, we select a vari-
ety of downstream tasks where we assume at least
one of the considered tools to be useful, and evalu-
ate performance in zero-shot settings (Section 4.2).
Beyond that, we also ensure that our approach does
not hurt the model’s core language modeling abili-
ties; we verify this by looking at perplexity on two
language modeling datasets (Section 4.3). Finally,
we investigate how the ability to learn using tools
is affected by model size (Section 4.4).
Toolformer