title: Adaptive Decoding via Latent Preference Optimization
summary: During language model decoding, it is known that using higher temperature
  sampling gives more creative responses, while lower temperatures are more
  factually accurate. However, such models are commonly applied to general
  instruction following, which involves both creative and fact seeking tasks,
  using a single fixed temperature across all examples and tokens. In this work,
  we introduce Adaptive Decoding, a layer added to the model to select the
  sampling temperature dynamically at inference time, at either the token or
  example level, in order to optimize performance. To learn its parameters we
  introduce Latent Preference Optimization (LPO) a general approach to train
  discrete latent variables such as choices of temperature. Our method
  outperforms all fixed decoding temperatures across a range of tasks that
  require different temperatures, including UltraFeedback, Creative Story
  Writing, and GSM8K.
paper_authors._id: 673890125ca44c7491b36202
paper_authors.name: Shehzaad Dhuliawala
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-19T16:29:29.350Z
paper_authors._id: 6325db12ddb9ff68f4698292
paper_authors.avatarUrl: /avatars/88e9f57a17bc9b41ddad508e7b1d2640.svg
paper_authors.fullname: Shehzaad Dhuliawala
paper_authors.type: user
paper_authors.user: shehzaadzd
paper_authors._id: 673890125ca44c7491b36203
paper_authors.name: Ilia Kulikov
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-19T16:29:34.928Z
paper_authors._id: 65d645913bc37244c55c39d5
paper_authors.avatarUrl: /avatars/508372876a654957650ee9429886d10c.svg
paper_authors.fullname: ilia kulikov
paper_authors.type: user
paper_authors.user: uralik
paper_authors._id: 673890125ca44c7491b36204
paper_authors.name: Ping Yu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-19T16:29:53.452Z
paper_authors._id: 646c004e31968a60a022590e
paper_authors.avatarUrl: /avatars/349e2ed4feef1be050fd9903dccec1e1.svg
paper_authors.fullname: Ping Yu
paper_authors.type: user
paper_authors.user: pingyu
paper_authors._id: 673890125ca44c7491b36205
paper_authors.name: Asli Celikyilmaz
paper_authors._id: 673890125ca44c7491b36206
paper_authors.name: Jason Weston
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-19T16:30:07.574Z
paper_authors._id: 62f023a36a027498eaa2f9cc
paper_authors.avatarUrl: /avatars/8ac1c5c74d0957e3c6cc94b3a7795c37.svg
paper_authors.fullname: Jason Weston
paper_authors.type: user
paper_authors.user: spermwhale
paper_authors._id: 673890125ca44c7491b36207
paper_authors.name: Sainbayar Sukhbaatar
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-19T16:30:15.503Z
paper_authors._id: 66a8611eb51510d82ed54231
paper_authors.avatarUrl: /avatars/ad559e774fee4914091b82c9831ae2a2.svg
paper_authors.fullname: Sainbayar Sukhbaatar
paper_authors.type: user
paper_authors.user: sainbar
paper_authors._id: 673890125ca44c7491b36208
paper_authors.name: Jack Lanchantin
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-20T10:09:12.355Z
paper_authors._id: 65dd30a6ea1a202d3cb469f7
paper_authors.avatarUrl: /avatars/14029def026380e6e20822d916b25a72.svg
paper_authors.fullname: Jack Lanchantin
paper_authors.type: user
paper_authors.user: jcklcn
paper_title: Adaptive Decoding via Latent Preference Optimization
paper_summary: During language model decoding, it is known that using higher temperature
  sampling gives more creative responses, while lower temperatures are more
  factually accurate. However, such models are commonly applied to general
  instruction following, which involves both creative and fact seeking tasks,
  using a single fixed temperature across all examples and tokens. In this work,
  we introduce Adaptive Decoding, a layer added to the model to select the
  sampling temperature dynamically at inference time, at either the token or
  example level, in order to optimize performance. To learn its parameters we
  introduce Latent Preference Optimization (LPO) a general approach to train
  discrete latent variables such as choices of temperature. Our method
  outperforms all fixed decoding temperatures across a range of tasks that
  require different temperatures, including UltraFeedback, Creative Story
  Writing, and GSM8K.
paper_ai_keywords: Adaptive Decoding Calibration temperature Inference time Token Example level Latent Preference Optimization (LPO) Discrete latent variables UltraFeedback Creative Story Writing GSM8K