title: ChatGPT-steered Editing Instructor for Customization of Abstractive
    Summarization
summary: Tailoring outputs of large language models, such as ChatGPT, to specific user
  needs remains a challenge despite their impressive generation quality. In this
  paper, we propose a tri-agent generation pipeline consisting of a generator, an
  instructor, and an editor to enhance the customization of generated outputs.
  The generator produces an initial output, the user-specific instructor
  generates editing instructions, and the editor generates a revised output
  aligned with user preferences. The inference-only large language model
  (ChatGPT) serves as both the generator and the editor, while a smaller model
  acts as the user-specific instructor to guide the generation process toward
  user needs. The instructor is trained using editor-steered reinforcement
  learning, leveraging feedback from the large-scale editor model to optimize
  instruction generation. Experimental results on two abstractive summarization
  datasets demonstrate the effectiveness of our approach in generating outputs
  that better fulfill user expectations.
paper_authors._id: 645454d0cd09ceba0e1596b7
paper_authors.name: Wen Xiao
paper_authors._id: 645454d0cd09ceba0e1596b8
paper_authors.name: Yujia Xie
paper_authors._id: 645454d0cd09ceba0e1596b9
paper_authors.name: Giuseppe Carenini
paper_authors._id: 645454d0cd09ceba0e1596ba
paper_authors.name: Pengcheng He
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-06-02T13:39:54.233Z
paper_authors._id: 5efd09cf49ed724c8a135868
paper_authors.avatarUrl: /avatars/af12bc94657979677a9f26183f0c9727.svg
paper_authors.fullname: Pengcheng He
paper_authors.type: user
paper_authors.user: DeBERTa
paper_title: ChatGPT-steered Editing Instructor for Customization of Abstractive
    Summarization
paper_summary: Tailoring outputs of large language models, such as ChatGPT, to specific user
  needs remains a challenge despite their impressive generation quality. In this
  paper, we propose a tri-agent generation pipeline consisting of a generator, an
  instructor, and an editor to enhance the customization of generated outputs.
  The generator produces an initial output, the user-specific instructor
  generates editing instructions, and the editor generates a revised output
  aligned with user preferences. The inference-only large language model
  (ChatGPT) serves as both the generator and the editor, while a smaller model
  acts as the user-specific instructor to guide the generation process toward
  user needs. The instructor is trained using editor-steered reinforcement
  learning, leveraging feedback from the large-scale editor model to optimize
  instruction generation. Experimental results on two abstractive summarization
  datasets demonstrate the effectiveness of our approach in generating outputs
  that better fulfill user expectations.
paper_ai_summary: A tri-agent pipeline for enhancing customizability of large language model outputs includes a generator, an instructor, and an editor, with the instructor being optimized through reinforcement learning guided by the editor.
paper_ai_keywords: large language models ChatGPT tri-agent generation pipeline generator instructor editor reinforcement learning editor-steered reinforcement learning abstractive summarization