4

Figure 3: This figure de-
scribes the generalization ac-
curacy of models to N + 1
digit addition after only train-
ing on up to N digit addition.
Consistent with past work, we
find that models show poor
length generalization on stan-
dard (fast) addition. How-
ever, when models use chain-
of-thought reasoning, gener-
alization reaches almost per-
fect accuracy after training on
only a few digits.

In order to avoid any such issues, we use the ByT5 family, which operates on the raw character/byte
value. A detailed description of the hyperparameters used in training can be found in Appendix A.

3.2 SUPERVISED TRAINING
CHAIN-OF-THOUGHTREASONING IS APOLICY IMPROVEMENTOPERATOR