Why Has Predicting Downstream Capabilities of Frontier AI Models with
  Scale Remained Elusive?
Predictable behavior from scaling advanced AI systems is an extremely
desirable property. Although a well-established literature exists on how
pretraining performance scales, the literature on how particular downstream
capabilities scale is significantly muddier. In this work, we take a step back
and ask: why has predicting specific downstream capabilities with scale
remained elusive? While many factors are certainly responsible, we identify a
new factor that makes modeling scaling behavior on widely used multiple-choice
question-answering benchmarks challenging. Using five model families and twelve
well-established multiple-choice benchmarks, we show that downstream
performance is computed from negative log likelihoods via a sequence of
transformations that progressively degrade the statistical relationship between
performance and scale. We then reveal the mechanism causing this degradation:
downstream metrics require comparing the correct choice against a small number
of specific incorrect choices, meaning accurately predicting downstream
capabilities requires predicting not just how probability mass concentrates on
the correct choice with scale, but also how probability mass fluctuates on
specific incorrect choices with scale. We empirically study how probability
mass on the correct choice co-varies with probability mass on incorrect choices
with increasing compute, suggesting that scaling laws for incorrect choices
might be achievable. Our work also explains why pretraining scaling laws are
commonly regarded as more predictable than downstream capabilities and
contributes towards establishing scaling-predictable evaluations of frontier AI
models.
66665ee6a1e7372885ecff45
Rylan Schaeffer
admin_assigned
2024-06-10T11:59:29.432Z
612abe088ef01bc4520e106c
/avatars/7576cedd550440c22e17f3d7a4d19787.svg
Rylan Schaeffer
user
RylanSchaeffer
66665ee6a1e7372885ecff46
Hailey Schoelkopf
admin_assigned
2024-06-10T11:59:36.891Z
62895a0215aeee85756062c4
https://cdn-avatars.huggingface.co/v1/production/uploads/1669665010552-62895a0215aeee85756062c4.jpeg
Hailey Schoelkopf
user
hails
66665ee6a1e7372885ecff47
Brando Miranda
admin_assigned
2024-06-10T11:59:43.189Z
60ec6453046c81764c7e06f0
https://cdn-avatars.huggingface.co/v1/production/uploads/60ec6453046c81764c7e06f0/gnKlFE5bkpvbDQNb2jrnV.png
Brando Miranda
user
brando
66665ee6a1e7372885ecff48
Gabriel Mukobi
admin_assigned
2024-06-10T11:59:49.605Z
640d68634b5450a208b37d70
https://cdn-avatars.huggingface.co/v1/production/uploads/1678599922732-noauth.jpeg
Gabriel Mukobi
user
gmukobi
66665ee6a1e7372885ecff49
Varun Madan
admin_assigned
2024-06-10T11:59:55.525Z
665e88de386cac3276f5ee87
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/QmMdDrapBpV5iBkUPZUHT.jpeg
Varun Madan
user
madanva
66665ee6a1e7372885ecff4a
Adam Ibrahim
admin_assigned
2024-06-10T12:00:04.456Z
648743f05aed4a7b0bf2e3a6
/avatars/e6f3b35085b1b8cb2c231516d1fea70a.svg
Adam Ibrahim
user
adami1
66665ee6a1e7372885ecff4b
Herbie Bradley
admin_assigned
2024-06-10T12:00:10.320Z
62f79f8147d782a6e28440f9
https://cdn-avatars.huggingface.co/v1/production/uploads/62f79f8147d782a6e28440f9/L-27MtgYpD1XQOBnx1gzL.jpeg
Herbie Bradley
user
HyperionHF
66665ee6a1e7372885ecff4c
Stella Biderman
admin_assigned
2024-06-10T12:00:16.633Z
60347d3660e3dd96631c9093
https://cdn-avatars.huggingface.co/v1/production/uploads/60347d3660e3dd96631c9093/B3fuZer5N04tZIAYrLnz4.jpeg
Stella Biderman
user
stellaathena
66665ee6a1e7372885ecff4d
Sanmi Koyejo
admin_assigned
2024-06-10T12:00:23.360Z
64931e7e2da595588288f161
https://cdn-avatars.huggingface.co/v1/production/uploads/64931e7e2da595588288f161/4jOhJOFsU7RVFMgGk5kO7.jpeg
Sanmi Koyejo
user
sanmikoyejo
Why Has Predicting Downstream Capabilities of Frontier AI Models with
  Scale Remained Elusive?
Predictable behavior from scaling advanced AI systems is an extremely
desirable property. Although a well-established literature exists on how
pretraining performance scales, the literature on how particular downstream
capabilities scale is significantly muddier. In this work, we take a step back
and ask: why has predicting specific downstream capabilities with scale
remained elusive? While many factors are certainly responsible, we identify a
new factor that makes modeling scaling behavior on widely used multiple-choice
question-answering benchmarks challenging. Using five model families and twelve
well-established multiple-choice benchmarks, we show that downstream
performance is computed from negative log likelihoods via a sequence of
transformations that progressively degrade the statistical relationship between
performance and scale. We then reveal the mechanism causing this degradation:
downstream metrics require comparing the correct choice against a small number
of specific incorrect choices, meaning accurately predicting downstream
capabilities requires predicting not just how probability mass concentrates on
the correct choice with scale, but also how probability mass fluctuates on
specific incorrect choices with scale. We empirically study how probability
mass on the correct choice co-varies with probability mass on incorrect choices
with increasing compute, suggesting that scaling laws for incorrect choices
might be achievable. Our work also explains why pretraining scaling laws are
commonly regarded as more predictable than downstream capabilities and
contributes towards establishing scaling-predictable evaluations of frontier AI
models.
Research identifies a new factor affecting predictability of specific downstream capabilities in advanced AI systems as they scale, focusing on the impact of probability distribution changes on correct and incorrect choices in multiple-choice benchmarks.
negative log likelihoods multiple-choice question-answering benchmarks scaling laws probability mass