title: RecurrentGPT: Interactive Generation of (Arbitrarily) Long Text
summary: The fixed-size context of Transformer makes GPT models incapable of
  generating arbitrarily long text. In this paper, we introduce RecurrentGPT, a
  language-based simulacrum of the recurrence mechanism in RNNs. RecurrentGPT is
  built upon a large language model (LLM) such as ChatGPT and uses natural
  language to simulate the Long Short-Term Memory mechanism in an LSTM. At each
  timestep, RecurrentGPT generates a paragraph of text and updates its
  language-based long-short term memory stored on the hard drive and the prompt,
  respectively. This recurrence mechanism enables RecurrentGPT to generate texts
  of arbitrary length without forgetting. Since human users can easily observe
  and edit the natural language memories, RecurrentGPT is interpretable and
  enables interactive generation of long text. RecurrentGPT is an initial step
  towards next-generation computer-assisted writing systems beyond local editing
  suggestions. In addition to producing AI-generated content (AIGC), we also
  demonstrate the possibility of using RecurrentGPT as an interactive fiction
  that directly interacts with consumers. We call this usage of generative models
  by ``AI As Contents'' (AIAC), which we believe is the next form of conventional
  AIGC. We further demonstrate the possibility of using RecurrentGPT to create
  personalized interactive fiction that directly interacts with readers instead
  of interacting with writers. More broadly, RecurrentGPT demonstrates the
  utility of borrowing ideas from popular model designs in cognitive science and
  deep learning for prompting LLMs. Our code is available at
  https://github.com/aiwaves-cn/RecurrentGPT and an online demo is available at
  https://www.aiwaves.org/recurrentgpt.
paper_authors._id: 646c18ee31968a60a022ddfe
paper_authors.name: Wangchunshu Zhou
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-23T08:59:07.337Z
paper_authors._id: 628c8598ef14f971b698107f
paper_authors.avatarUrl: /avatars/3a4ad87e6b5f9e836a1160d869df1447.svg
paper_authors.fullname: Zhou
paper_authors.type: user
paper_authors.user: Wangchunshu
paper_authors._id: 646c18ee31968a60a022ddff
paper_authors.name: Yuchen Eleanor Jiang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-05-23T08:58:04.546Z
paper_authors._id: 62c872ff5a87d76d88fdba75
paper_authors.avatarUrl: /avatars/d4696529d1d135e5e2774eb74776ec58.svg
paper_authors.fullname: Yuchen Eleanor Jiang
paper_authors.type: user
paper_authors.user: eleanorjiang
paper_authors._id: 646c18ee31968a60a022de00
paper_authors.name: Peng Cui
paper_authors._id: 646c18ee31968a60a022de01
paper_authors.name: Tiannan Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-24T15:30:47.718Z
paper_authors._id: 632bfaebea6e62428ab0e9c2
paper_authors.avatarUrl: /avatars/344aaf371bbba9aea091b12741c451e5.svg
paper_authors.fullname: Tiannan Wang
paper_authors.type: user
paper_authors.user: WTNswaggy
paper_authors._id: 646c18ee31968a60a022de02
paper_authors.name: Zhenxin Xiao
paper_authors._id: 646c18ee31968a60a022de03
paper_authors.name: Yifan Hou
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-23T12:22:42.575Z
paper_authors._id: 63594791d21ba0962c81d6fc
paper_authors.avatarUrl: /avatars/1790c2cf741d0dc5b4b36c310d5afea2.svg
paper_authors.fullname: Yifan Hou
paper_authors.type: user
paper_authors.user: yyyyifan
paper_authors._id: 646c18ee31968a60a022de04
paper_authors.name: Ryan Cotterell
paper_authors._id: 646c18ee31968a60a022de05
paper_authors.name: Mrinmaya Sachan
paper_title: RecurrentGPT: Interactive Generation of (Arbitrarily) Long Text
paper_summary: The fixed-size context of Transformer makes GPT models incapable of
  generating arbitrarily long text. In this paper, we introduce RecurrentGPT, a
  language-based simulacrum of the recurrence mechanism in RNNs. RecurrentGPT is
  built upon a large language model (LLM) such as ChatGPT and uses natural
  language to simulate the Long Short-Term Memory mechanism in an LSTM. At each
  timestep, RecurrentGPT generates a paragraph of text and updates its
  language-based long-short term memory stored on the hard drive and the prompt,
  respectively. This recurrence mechanism enables RecurrentGPT to generate texts
  of arbitrary length without forgetting. Since human users can easily observe
  and edit the natural language memories, RecurrentGPT is interpretable and
  enables interactive generation of long text. RecurrentGPT is an initial step
  towards next-generation computer-assisted writing systems beyond local editing
  suggestions. In addition to producing AI-generated content (AIGC), we also
  demonstrate the possibility of using RecurrentGPT as an interactive fiction
  that directly interacts with consumers. We call this usage of generative models
  by ``AI As Contents'' (AIAC), which we believe is the next form of conventional
  AIGC. We further demonstrate the possibility of using RecurrentGPT to create
  personalized interactive fiction that directly interacts with readers instead
  of interacting with writers. More broadly, RecurrentGPT demonstrates the
  utility of borrowing ideas from popular model designs in cognitive science and
  deep learning for prompting LLMs. Our code is available at
  https://github.com/aiwaves-cn/RecurrentGPT and an online demo is available at
  https://www.aiwaves.org/recurrentgpt.
paper_ai_keywords: Transformer RecurrentGPT Large Language Model (LLM) ChatGPT Long Short-Term Memory (LSTM) natural language memories interactive generation AI-generated content (AIGC) AI As Contents (AIAC) cognitive science deep Learning prompting LLMs