more start to exhibit a correlation between task accuracy and
occurrence of task-relevant terms which is not present in
prior checkpoints and is largely absent from smaller models.
Pythia- A Suite for Analyzing Large Language Models Across Training and Scaling