title: Why Has Predicting Downstream Capabilities of Frontier AI Models with
    Scale Remained Elusive?
summary: Predictable behavior from scaling advanced AI systems is an extremely
  desirable property. Although a well-established literature exists on how
  pretraining performance scales, the literature on how particular downstream
  capabilities scale is significantly muddier. In this work, we take a step back
  and ask: why has predicting specific downstream capabilities with scale
  remained elusive? While many factors are certainly responsible, we identify a
  new factor that makes modeling scaling behavior on widely used multiple-choice
  question-answering benchmarks challenging. Using five model families and twelve
  well-established multiple-choice benchmarks, we show that downstream
  performance is computed from negative log likelihoods via a sequence of
  transformations that progressively degrade the statistical relationship between
  performance and scale. We then reveal the mechanism causing this degradation:
  downstream metrics require comparing the correct choice against a small number
  of specific incorrect choices, meaning accurately predicting downstream
  capabilities requires predicting not just how probability mass concentrates on
  the correct choice with scale, but also how probability mass fluctuates on
  specific incorrect choices with scale. We empirically study how probability
  mass on the correct choice co-varies with probability mass on incorrect choices
  with increasing compute, suggesting that scaling laws for incorrect choices
  might be achievable. Our work also explains why pretraining scaling laws are
  commonly regarded as more predictable than downstream capabilities and
  contributes towards establishing scaling-predictable evaluations of frontier AI
  models.
paper_authors._id: 66665ee6a1e7372885ecff45
paper_authors.name: Rylan Schaeffer
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-10T11:59:29.432Z
paper_authors._id: 612abe088ef01bc4520e106c
paper_authors.avatarUrl: /avatars/7576cedd550440c22e17f3d7a4d19787.svg
paper_authors.fullname: Rylan Schaeffer
paper_authors.type: user
paper_authors.user: RylanSchaeffer
paper_authors._id: 66665ee6a1e7372885ecff46
paper_authors.name: Hailey Schoelkopf
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-10T11:59:36.891Z
paper_authors._id: 62895a0215aeee85756062c4
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1669665010552-62895a0215aeee85756062c4.jpeg
paper_authors.fullname: Hailey Schoelkopf
paper_authors.type: user
paper_authors.user: hails
paper_authors._id: 66665ee6a1e7372885ecff47
paper_authors.name: Brando Miranda
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-10T11:59:43.189Z
paper_authors._id: 60ec6453046c81764c7e06f0
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/60ec6453046c81764c7e06f0/gnKlFE5bkpvbDQNb2jrnV.png
paper_authors.fullname: Brando Miranda
paper_authors.type: user
paper_authors.user: brando
paper_authors._id: 66665ee6a1e7372885ecff48
paper_authors.name: Gabriel Mukobi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-10T11:59:49.605Z
paper_authors._id: 640d68634b5450a208b37d70
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1678599922732-noauth.jpeg
paper_authors.fullname: Gabriel Mukobi
paper_authors.type: user
paper_authors.user: gmukobi
paper_authors._id: 66665ee6a1e7372885ecff49
paper_authors.name: Varun Madan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-10T11:59:55.525Z
paper_authors._id: 665e88de386cac3276f5ee87
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/QmMdDrapBpV5iBkUPZUHT.jpeg
paper_authors.fullname: Varun Madan
paper_authors.type: user
paper_authors.user: madanva
paper_authors._id: 66665ee6a1e7372885ecff4a
paper_authors.name: Adam Ibrahim
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-10T12:00:04.456Z
paper_authors._id: 648743f05aed4a7b0bf2e3a6
paper_authors.avatarUrl: /avatars/e6f3b35085b1b8cb2c231516d1fea70a.svg
paper_authors.fullname: Adam Ibrahim
paper_authors.type: user
paper_authors.user: adami1
paper_authors._id: 66665ee6a1e7372885ecff4b
paper_authors.name: Herbie Bradley
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-10T12:00:10.320Z
paper_authors._id: 62f79f8147d782a6e28440f9
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/62f79f8147d782a6e28440f9/L-27MtgYpD1XQOBnx1gzL.jpeg
paper_authors.fullname: Herbie Bradley
paper_authors.type: user
paper_authors.user: HyperionHF
paper_authors._id: 66665ee6a1e7372885ecff4c
paper_authors.name: Stella Biderman
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-10T12:00:16.633Z
paper_authors._id: 60347d3660e3dd96631c9093
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/60347d3660e3dd96631c9093/B3fuZer5N04tZIAYrLnz4.jpeg
paper_authors.fullname: Stella Biderman
paper_authors.type: user
paper_authors.user: stellaathena
paper_authors._id: 66665ee6a1e7372885ecff4d
paper_authors.name: Sanmi Koyejo
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-10T12:00:23.360Z
paper_authors._id: 64931e7e2da595588288f161
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64931e7e2da595588288f161/4jOhJOFsU7RVFMgGk5kO7.jpeg
paper_authors.fullname: Sanmi Koyejo
paper_authors.type: user
paper_authors.user: sanmikoyejo
paper_title: Why Has Predicting Downstream Capabilities of Frontier AI Models with
    Scale Remained Elusive?
paper_summary: Predictable behavior from scaling advanced AI systems is an extremely
  desirable property. Although a well-established literature exists on how
  pretraining performance scales, the literature on how particular downstream
  capabilities scale is significantly muddier. In this work, we take a step back
  and ask: why has predicting specific downstream capabilities with scale
  remained elusive? While many factors are certainly responsible, we identify a
  new factor that makes modeling scaling behavior on widely used multiple-choice
  question-answering benchmarks challenging. Using five model families and twelve
  well-established multiple-choice benchmarks, we show that downstream
  performance is computed from negative log likelihoods via a sequence of
  transformations that progressively degrade the statistical relationship between
  performance and scale. We then reveal the mechanism causing this degradation:
  downstream metrics require comparing the correct choice against a small number
  of specific incorrect choices, meaning accurately predicting downstream
  capabilities requires predicting not just how probability mass concentrates on
  the correct choice with scale, but also how probability mass fluctuates on
  specific incorrect choices with scale. We empirically study how probability
  mass on the correct choice co-varies with probability mass on incorrect choices
  with increasing compute, suggesting that scaling laws for incorrect choices
  might be achievable. Our work also explains why pretraining scaling laws are
  commonly regarded as more predictable than downstream capabilities and
  contributes towards establishing scaling-predictable evaluations of frontier AI
  models.
paper_ai_summary: Research identifies a new factor affecting predictability of specific downstream capabilities in advanced AI systems as they scale, focusing on the impact of probability distribution changes on correct and incorrect choices in multiple-choice benchmarks.
paper_ai_keywords: negative log likelihoods multiple-choice question-answering benchmarks scaling laws probability mass