title: OmniPred: Language Models as Universal Regressors
summary: Over the broad landscape of experimental design, regression has been a
  powerful tool to accurately predict the outcome metrics of a system or model
  given a set of parameters, but has been traditionally restricted to methods
  which are only applicable to a specific task. In this paper, we propose
  OmniPred, a framework for training language models as universal end-to-end
  regressors over (x,y) evaluation data from diverse real world experiments.
  Using data sourced from Google Vizier, one of the largest blackbox optimization
  databases in the world, our extensive experiments demonstrate that through only
  textual representations of mathematical parameters and values, language models
  are capable of very precise numerical regression, and if given the opportunity
  to train over multiple tasks, can significantly outperform traditional
  regression models.
paper_authors._id: 65d8116a5959cf8b839af381
paper_authors.name: Xingyou Song
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-23T12:39:45.864Z
paper_authors._id: 64c93c8c4524c2aea71d2ec6
paper_authors.avatarUrl: /avatars/1961fb6209182bf4954778dc3d97dde5.svg
paper_authors.fullname: Xingyou Song
paper_authors.type: user
paper_authors.user: Srizzle
paper_authors._id: 65d8116a5959cf8b839af382
paper_authors.name: Oscar Li
paper_authors._id: 65d8116a5959cf8b839af383
paper_authors.name: Chansoo Lee
paper_authors._id: 65d8116a5959cf8b839af384
paper_authors.name: Bangding
paper_authors._id: 65d8116a5959cf8b839af385
paper_authors.name: Yang
paper_authors._id: 65d8116a5959cf8b839af386
paper_authors.name: Daiyi Peng
paper_authors._id: 65d8116a5959cf8b839af387
paper_authors.name: Sagi Perel
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-23T12:40:17.934Z
paper_authors._id: 63b0e82243aaf9276fed1d7f
paper_authors.avatarUrl: /avatars/ed6df77f106aabd7fa82e0602264d1df.svg
paper_authors.fullname: Sagi Perel
paper_authors.type: user
paper_authors.user: sagipe
paper_authors._id: 65d8116a5959cf8b839af388
paper_authors.name: Yutian Chen
paper_title: OmniPred: Language Models as Universal Regressors
paper_summary: Over the broad landscape of experimental design, regression has been a
  powerful tool to accurately predict the outcome metrics of a system or model
  given a set of parameters, but has been traditionally restricted to methods
  which are only applicable to a specific task. In this paper, we propose
  OmniPred, a framework for training language models as universal end-to-end
  regressors over (x,y) evaluation data from diverse real world experiments.
  Using data sourced from Google Vizier, one of the largest blackbox optimization
  databases in the world, our extensive experiments demonstrate that through only
  textual representations of mathematical parameters and values, language models
  are capable of very precise numerical regression, and if given the opportunity
  to train over multiple tasks, can significantly outperform traditional
  regression models.
paper_ai_summary: OmniPred, a framework training language models as universal end-to-end regressors, outperforms traditional regression models using textual representations of parameters and values from diverse experimental data.
paper_ai_keywords: OmniPred language models end-to-end regressors Google Vizier blackbox optimization numerical regression