title: On the Transformations across Reward Model, Parameter Update, and
    In-Context Prompt
summary: Despite the general capabilities of pre-trained large language models (LLMs),
  they still need further adaptation to better serve practical applications. In
  this paper, we demonstrate the interchangeability of three popular and distinct
  adaptation tools: parameter updating, reward modeling, and in-context
  prompting. This interchangeability establishes a triangular framework with six
  transformation directions, each of which facilitates a variety of applications.
  Our work offers a holistic view that unifies numerous existing studies and
  suggests potential research directions. We envision our work as a useful
  roadmap for future research on LLMs.
paper_authors._id: 667bbd1f20887e69a0767249
paper_authors.name: Deng Cai
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-26T12:57:48.845Z
paper_authors._id: 646735a1946476c5d214e5b2
paper_authors.avatarUrl: /avatars/23392fda23fa6288fcb309f7eaa73472.svg
paper_authors.fullname: Deng Cai
paper_authors.type: user
paper_authors.user: jcyk
paper_authors._id: 667bbd1f20887e69a076724a
paper_authors.name: Huayang Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-26T12:57:57.304Z
paper_authors._id: 64672e286192d3914221d633
paper_authors.avatarUrl: /avatars/a26908ebcbb74feaf44fe83a44fef8ac.svg
paper_authors.fullname: Huayang
paper_authors.type: user
paper_authors.user: huayangli
paper_authors._id: 667bbd1f20887e69a076724b
paper_authors.name: Tingchen Fu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-26T12:58:06.805Z
paper_authors._id: 640ad17a1ee054d66a74783e
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/640ad17a1ee054d66a74783e/u0PjIkyC-9HkGzEyUQ7JN.jpeg
paper_authors.fullname: Tingchen Fu
paper_authors.type: user
paper_authors.user: TingchenFu
paper_authors._id: 667bbd1f20887e69a076724c
paper_authors.name: Siheng Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-26T12:58:12.922Z
paper_authors._id: 62f71b23d3bdacb7eec43b1f
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/62f71b23d3bdacb7eec43b1f/6RNVG9zug9tutjo5i8T4C.jpeg
paper_authors.fullname: SihengLi
paper_authors.type: user
paper_authors.user: SihengLi
paper_authors._id: 667bbd1f20887e69a076724d
paper_authors.name: Weiwen Xu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-26T12:58:23.686Z
paper_authors._id: 64118689756b9e455c7eac62
paper_authors.avatarUrl: /avatars/cdb3da22593facf545a0bafbf548b07e.svg
paper_authors.fullname: Xu Weiwen
paper_authors.type: user
paper_authors.user: xww033
paper_authors._id: 667bbd1f20887e69a076724e
paper_authors.name: Shuaiyi Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-26T12:58:34.454Z
paper_authors._id: 64d708e45e5f05485ce5fbd8
paper_authors.avatarUrl: /avatars/3c5389a5ffecbc05583004c627b21b6c.svg
paper_authors.fullname: Li Shuaiyi
paper_authors.type: user
paper_authors.user: Syon-Li
paper_authors._id: 667bbd1f20887e69a076724f
paper_authors.name: Bowen Cao
paper_authors._id: 667bbd1f20887e69a0767250
paper_authors.name: Zhisong Zhang
paper_authors._id: 667bbd1f20887e69a0767251
paper_authors.name: Xinting Huang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-06-30T20:57:45.272Z
paper_authors._id: 63c227a6c58fcfeac18dec07
paper_authors.avatarUrl: /avatars/47df92c937cfaf8d726c7e28cf352eb5.svg
paper_authors.fullname: tim huang
paper_authors.type: user
paper_authors.user: timmyhhh
paper_authors._id: 667bbd1f20887e69a0767252
paper_authors.name: Leyang Cui
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-26T12:59:17.570Z
paper_authors._id: 646db809e4fca2e115db428c
paper_authors.avatarUrl: /avatars/17d77f59885db13d4f9c10a9b1245131.svg
paper_authors.fullname: Leyang Cui
paper_authors.type: user
paper_authors.user: nealcly
paper_authors._id: 667bbd1f20887e69a0767253
paper_authors.name: Yan Wang
paper_authors._id: 667bbd1f20887e69a0767254
paper_authors.name: Lemao Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-26T12:59:27.508Z
paper_authors._id: 64462d77efa0723f3946356b
paper_authors.avatarUrl: /avatars/f9757030d82c69aef933309e0c83ccd0.svg
paper_authors.fullname: Lemao Liu
paper_authors.type: user
paper_authors.user: lemaoliu
paper_authors._id: 667bbd1f20887e69a0767255
paper_authors.name: Taro Watanabe
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-26T12:59:32.954Z
paper_authors._id: 65a74e2aae68caef0b82c27e
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/65a74e2aae68caef0b82c27e/tnPaCv2PeDfSX1G7v3QrQ.jpeg
paper_authors.fullname: Taro Watanabe
paper_authors.type: user
paper_authors.user: tarowatanabe
paper_authors._id: 667bbd1f20887e69a0767256
paper_authors.name: Shuming Shi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-26T12:59:50.079Z
paper_authors._id: 62eb8e9eaf9bed70f5ad5a7e
paper_authors.avatarUrl: /avatars/971763f3b724d63bc64b7d3599cfc753.svg
paper_authors.fullname: Shuming Shi
paper_authors.type: user
paper_authors.user: Shuming
paper_title: On the Transformations across Reward Model, Parameter Update, and
    In-Context Prompt
paper_summary: Despite the general capabilities of pre-trained large language models (LLMs),
  they still need further adaptation to better serve practical applications. In
  this paper, we demonstrate the interchangeability of three popular and distinct
  adaptation tools: parameter updating, reward modeling, and in-context
  prompting. This interchangeability establishes a triangular framework with six
  transformation directions, each of which facilitates a variety of applications.
  Our work offers a holistic view that unifies numerous existing studies and
  suggests potential research directions. We envision our work as a useful
  roadmap for future research on LLMs.
paper_ai_summary: Three adaptation tools—parameter updating, reward modeling, and in-context prompting—are interchangeable in LLMs, forming a framework that supports multiple applications and guides future research.
paper_ai_keywords: pre-trained large language models parameter updating reward modeling in-context prompting