Inference When generating text with M after
ﬁnetuning with our approach, we perform regular
decoding until M produces the “→” token, indicat-
ing that it next expects the response for an API call.
At this point, we interrupt the decoding process,
call the appropriate API to get a response, and con-
tinue the decoding process after inserting both the
response and the </API> token.

3 Tools
Toolformer