While the aforementioned works experiment with various
model sizes, it is not yet studied when during training and at
what model sizes this effect occurs. We further investigate
this phenomenon across model checkpoints and model sizes
by adapting arithmetic tasks of multiplication and addition
(Razeghi et al., 2022) and a QA task (Kandpal et al., 2022)
Pythia- A Suite for Analyzing Large Language Models Across Training and Scaling