similar self-learning approaches, this phenomenon caused the training process to go awry within just
a few steps (Zelikman et al., 2022; Jung et al., 2023; Bai et al., 2022; Huang et al., 2022). In order to
mitigate the effects of error avalanching, SECToR utilizes several consistency checks largely based
on the following idea: we ask the model to generate a large number of “equivalent” variations of
any given question and then ask the model to (independently) solve each variation. If the answers
are largely consistent, we accept these answers into the training data. Otherwise, SECToR discards
the answers. These consistency checks are essential for SECToR to minimize the introduction of
incorrect data into the model’s self-training loop.
CHAIN-OF-THOUGHTREASONING IS APOLICY IMPROVEMENTOPERATOR