title: LLMs + Persona-Plug = Personalized LLMs
summary: Personalization plays a critical role in numerous language tasks and
  applications, since users with the same requirements may prefer diverse outputs
  based on their individual interests. This has led to the development of various
  personalized approaches aimed at adapting large language models (LLMs) to
  generate customized outputs aligned with user preferences. Some of them involve
  fine-tuning a unique personalized LLM for each user, which is too expensive for
  widespread application. Alternative approaches introduce personalization
  information in a plug-and-play manner by retrieving the user's relevant
  historical texts as demonstrations. However, this retrieval-based strategy may
  break the continuity of the user history and fail to capture the user's overall
  styles and patterns, hence leading to sub-optimal performance. To address these
  challenges, we propose a novel personalized LLM model, . It constructs a
  user-specific embedding for each individual by modeling all her historical
  contexts through a lightweight plug-in user embedder module. By attaching this
  embedding to the task input, LLMs can better understand and capture user habits
  and preferences, thereby producing more personalized outputs without tuning
  their own parameters. Extensive experiments on various tasks in the language
  model personalization (LaMP) benchmark demonstrate that the proposed model
  significantly outperforms existing personalized LLM approaches.
paper_authors._id: 66eb8c29d57b0fc0c43521dd
paper_authors.name: Jiongnan Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-19T08:20:45.400Z
paper_authors._id: 65dd88b71f7352669d65f4f5
paper_authors.avatarUrl: /avatars/0cef87a5a40ddbc5530b31991862de28.svg
paper_authors.fullname: jiongnan liu
paper_authors.type: user
paper_authors.user: liujiongnan
paper_authors._id: 66eb8c29d57b0fc0c43521de
paper_authors.name: Yutao Zhu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-19T08:20:51.799Z
paper_authors._id: 625e62452a7279d3c77b5c38
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/625e62452a7279d3c77b5c38/zJINew6U4_Gup4WTobb-0.jpeg
paper_authors.fullname: Yutao Zhu
paper_authors.type: user
paper_authors.user: yutaozhu94
paper_authors._id: 66eb8c29d57b0fc0c43521df
paper_authors.name: Shuting Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-19T08:21:02.553Z
paper_authors._id: 656d68699dcedd16d5596d26
paper_authors.avatarUrl: /avatars/3486af06cc2c1562e09b04bb03360912.svg
paper_authors.fullname: Shu-Ting Wang
paper_authors.type: user
paper_authors.user: stingw
paper_authors._id: 66eb8c29d57b0fc0c43521e0
paper_authors.name: Xiaochi Wei
paper_authors._id: 66eb8c29d57b0fc0c43521e1
paper_authors.name: Erxue Min
paper_authors._id: 66eb8c29d57b0fc0c43521e2
paper_authors.name: Yu Lu
paper_authors._id: 66eb8c29d57b0fc0c43521e3
paper_authors.name: Shuaiqiang Wang
paper_authors._id: 66eb8c29d57b0fc0c43521e4
paper_authors.name: Dawei Yin
paper_authors._id: 66eb8c29d57b0fc0c43521e5
paper_authors.name: Zhicheng Dou
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-09-23T06:58:33.624Z
paper_authors._id: 66f0bf59e9d50ec57febf751
paper_authors.avatarUrl: /avatars/be97941e60064e5dd806c6fe9db3c537.svg
paper_authors.fullname: Zhicheng Dou
paper_authors.type: user
paper_authors.user: douzc
paper_title: LLMs + Persona-Plug = Personalized LLMs
paper_summary: Personalization plays a critical role in numerous language tasks and
  applications, since users with the same requirements may prefer diverse outputs
  based on their individual interests. This has led to the development of various
  personalized approaches aimed at adapting large language models (LLMs) to
  generate customized outputs aligned with user preferences. Some of them involve
  fine-tuning a unique personalized LLM for each user, which is too expensive for
  widespread application. Alternative approaches introduce personalization
  information in a plug-and-play manner by retrieving the user's relevant
  historical texts as demonstrations. However, this retrieval-based strategy may
  break the continuity of the user history and fail to capture the user's overall
  styles and patterns, hence leading to sub-optimal performance. To address these
  challenges, we propose a novel personalized LLM model, . It constructs a
  user-specific embedding for each individual by modeling all her historical
  contexts through a lightweight plug-in user embedder module. By attaching this
  embedding to the task input, LLMs can better understand and capture user habits
  and preferences, thereby producing more personalized outputs without tuning
  their own parameters. Extensive experiments on various tasks in the language
  model personalization (LaMP) benchmark demonstrate that the proposed model
  significantly outperforms existing personalized LLM approaches.
paper_ai_summary: A novel model constructs user-specific embeddings using a lightweight plug-in module to personalize LLM outputs without fine-tuning, improving performance on various tasks.
paper_ai_keywords: large language models (LLMs) personalized LLM user-specific embedding plug-in user embedder module language model personalization (LaMP) benchmark