ally, we save log-spaced checkpoints early in training at
iterations {1, 2, 4, 8, 16, 32, 64, 128, 256, 512}. This gives
a total of 154 checkpoints per model, far more than any
Pythia- A Suite for Analyzing Large Language Models Across Training and Scaling