All training data is self-
generated by the model

Until self training fails

Figure 1: SECToR begins with a supervised fine-tuning phase in which the model undergoes training
with curriculum learning, mastering simpler addition problems before progressing to more challeng-
ing ones. SECToR begins self-training when it generalizes perfectly to N + 1 digit addition (after
having been trained only on 1 through N digit addition) when equipped with chain-of-thought rea-
soning. Empirically, this occurred at 7 digits for the 582M parameter model. During self-training,
the training process largely mirrors the supervised training phase, except that all training data is au-
tonomously generated by the model using chain-of-thought reasoning and self-consistency checks
without any external verification of correctness (see Figure 12).
CHAIN-OF-THOUGHTREASONING IS APOLICY IMPROVEMENTOPERATOR