34.9

–

All AC NC %
0.0
18.9
18.9
8.5
19.3 17.1 19.9
26.3 26.5
6.6
99.3
26.3 26.4
100.0
–

Table 9: Toolformer results on the T-REx subset of
LAMA and on WebQS for different values of k used
during decoding. Numbers shown are overall perfor-
mance (All), performance on the subset where the
model decides to make an API call (AC) and all re-
maining examples (NC), as well as the percentage of
examples for which the model decides to call an API
(%).

“Fast train success” in the fourth example that does
not give any relevant information but still reduces
perplexity. However, some amount of noise in the
API calls that are not ﬁltered can actually be useful
as it forces the model ﬁnetuned on C∗ to not always
blindly follow the results of each call it makes.

6 Related Work
Toolformer