title: PEARL: Personalizing Large Language Model Writing Assistants with
    Generation-Calibrated Retrievers
summary: Powerful large language models have facilitated the development of writing
  assistants that promise to significantly improve the quality and efficiency of
  composition and communication. However, a barrier to effective assistance is
  the lack of personalization in LLM outputs to the author's communication style
  and specialized knowledge. In this paper, we address this challenge by
  proposing PEARL, a retrieval-augmented LLM writing assistant personalized with
  a generation-calibrated retriever. Our retriever is trained to select historic
  user-authored documents for prompt augmentation, such that they are likely to
  best personalize LLM generations for a user request. We propose two key
  novelties for training our retriever: 1) A training data selection method that
  identifies user requests likely to benefit from personalization and documents
  that provide that benefit; and 2) A scale-calibrating KL-divergence objective
  that ensures that our retriever closely tracks the benefit of a document for
  personalized generation. We demonstrate the effectiveness of PEARL in
  generating personalized workplace social media posts and Reddit comments.
  Finally, we showcase the potential of a generation-calibrated retriever to
  double as a performance predictor and further improve low-quality generations
  via LLM chaining.
paper_authors._id: 6555ab778094dce8237364d2
paper_authors.name: Sheshera Mysore
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-16T10:06:10.703Z
paper_authors._id: 62258434518cb976dadeaabd
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1646747907864-62258434518cb976dadeaabd.jpeg
paper_authors.fullname: Sheshera Mysore
paper_authors.type: user
paper_authors.user: Sheshera
paper_authors._id: 6555ab778094dce8237364d3
paper_authors.name: Zhuoran Lu
paper_authors._id: 6555ab778094dce8237364d4
paper_authors.name: Mengting Wan
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-03-27T08:46:39.220Z
paper_authors._id: 660328d9738bdf26e72dd454
paper_authors.avatarUrl: /avatars/7af51be1cc1dfaaa3a78e0b546d97cf8.svg
paper_authors.fullname: Mengting Wan
paper_authors.type: user
paper_authors.user: mengtingwan
paper_authors._id: 6555ab778094dce8237364d5
paper_authors.name: Longqi Yang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-16T10:06:39.703Z
paper_authors._id: 64d048b197ca59bcf726bdf2
paper_authors.avatarUrl: /avatars/6384aa7558382cad60ee94d9e08d6182.svg
paper_authors.fullname: Longqi Yang
paper_authors.type: user
paper_authors.user: ylongqi
paper_authors._id: 6555ab778094dce8237364d6
paper_authors.name: Steve Menezes
paper_authors._id: 6555ab778094dce8237364d7
paper_authors.name: Tina Baghaee
paper_authors._id: 6555ab778094dce8237364d8
paper_authors.name: Emmanuel Barajas Gonzalez
paper_authors._id: 6555ab778094dce8237364d9
paper_authors.name: Jennifer Neville
paper_authors._id: 6555ab778094dce8237364da
paper_authors.name: Tara Safavi
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2023-11-16T05:41:12.149Z
paper_authors._id: 63ffc744cafc9d54985fdf4f
paper_authors.avatarUrl: /avatars/0820cad83df012182cdf4094a7dcf744.svg
paper_authors.fullname: Tara Safavi
paper_authors.type: user
paper_authors.user: tararootcake
paper_title: PEARL: Personalizing Large Language Model Writing Assistants with
    Generation-Calibrated Retrievers
paper_summary: Powerful large language models have facilitated the development of writing
  assistants that promise to significantly improve the quality and efficiency of
  composition and communication. However, a barrier to effective assistance is
  the lack of personalization in LLM outputs to the author's communication style
  and specialized knowledge. In this paper, we address this challenge by
  proposing PEARL, a retrieval-augmented LLM writing assistant personalized with
  a generation-calibrated retriever. Our retriever is trained to select historic
  user-authored documents for prompt augmentation, such that they are likely to
  best personalize LLM generations for a user request. We propose two key
  novelties for training our retriever: 1) A training data selection method that
  identifies user requests likely to benefit from personalization and documents
  that provide that benefit; and 2) A scale-calibrating KL-divergence objective
  that ensures that our retriever closely tracks the benefit of a document for
  personalized generation. We demonstrate the effectiveness of PEARL in
  generating personalized workplace social media posts and Reddit comments.
  Finally, we showcase the potential of a generation-calibrated retriever to
  double as a performance predictor and further improve low-quality generations
  via LLM chaining.
paper_ai_summary: PEARL, a retrieval-augmented writing assistant, personalizes large language model outputs for users by selecting relevant documents, improving the quality of workplace social media posts and Reddit comments.
paper_ai_keywords: large language models writing assistants personalization retriever prompt augmentation training data selection KL-divergence objective generation-calibrated retriever low-quality generations LLM chaining