RecurrentGPT: Interactive Generation of (Arbitrarily) Long Text
The fixed-size context of Transformer makes GPT models incapable of
generating arbitrarily long text. In this paper, we introduce RecurrentGPT, a
language-based simulacrum of the recurrence mechanism in RNNs. RecurrentGPT is
built upon a large language model (LLM) such as ChatGPT and uses natural
language to simulate the Long Short-Term Memory mechanism in an LSTM. At each
timestep, RecurrentGPT generates a paragraph of text and updates its
language-based long-short term memory stored on the hard drive and the prompt,
respectively. This recurrence mechanism enables RecurrentGPT to generate texts
of arbitrary length without forgetting. Since human users can easily observe
and edit the natural language memories, RecurrentGPT is interpretable and
enables interactive generation of long text. RecurrentGPT is an initial step
towards next-generation computer-assisted writing systems beyond local editing
suggestions. In addition to producing AI-generated content (AIGC), we also
demonstrate the possibility of using RecurrentGPT as an interactive fiction
that directly interacts with consumers. We call this usage of generative models
by ``AI As Contents'' (AIAC), which we believe is the next form of conventional
AIGC. We further demonstrate the possibility of using RecurrentGPT to create
personalized interactive fiction that directly interacts with readers instead
of interacting with writers. More broadly, RecurrentGPT demonstrates the
utility of borrowing ideas from popular model designs in cognitive science and
deep learning for prompting LLMs. Our code is available at
https://github.com/aiwaves-cn/RecurrentGPT and an online demo is available at
https://www.aiwaves.org/recurrentgpt.
646c18ee31968a60a022ddfe
Wangchunshu Zhou
claimed_verified
2023-05-23T08:59:07.337Z
628c8598ef14f971b698107f
/avatars/3a4ad87e6b5f9e836a1160d869df1447.svg
Zhou
user
Wangchunshu
646c18ee31968a60a022ddff
Yuchen Eleanor Jiang
admin_assigned
2023-05-23T08:58:04.546Z
62c872ff5a87d76d88fdba75
/avatars/d4696529d1d135e5e2774eb74776ec58.svg
Yuchen Eleanor Jiang
user
eleanorjiang
646c18ee31968a60a022de00
Peng Cui
646c18ee31968a60a022de01
Tiannan Wang
claimed_verified
2023-05-24T15:30:47.718Z
632bfaebea6e62428ab0e9c2
/avatars/344aaf371bbba9aea091b12741c451e5.svg
Tiannan Wang
user
WTNswaggy
646c18ee31968a60a022de02
Zhenxin Xiao
646c18ee31968a60a022de03
Yifan Hou
claimed_verified
2023-05-23T12:22:42.575Z
63594791d21ba0962c81d6fc
/avatars/1790c2cf741d0dc5b4b36c310d5afea2.svg
Yifan Hou
user
yyyyifan
646c18ee31968a60a022de04
Ryan Cotterell
646c18ee31968a60a022de05
Mrinmaya Sachan
RecurrentGPT: Interactive Generation of (Arbitrarily) Long Text
The fixed-size context of Transformer makes GPT models incapable of
generating arbitrarily long text. In this paper, we introduce RecurrentGPT, a
language-based simulacrum of the recurrence mechanism in RNNs. RecurrentGPT is
built upon a large language model (LLM) such as ChatGPT and uses natural
language to simulate the Long Short-Term Memory mechanism in an LSTM. At each
timestep, RecurrentGPT generates a paragraph of text and updates its
language-based long-short term memory stored on the hard drive and the prompt,
respectively. This recurrence mechanism enables RecurrentGPT to generate texts
of arbitrary length without forgetting. Since human users can easily observe
and edit the natural language memories, RecurrentGPT is interpretable and
enables interactive generation of long text. RecurrentGPT is an initial step
towards next-generation computer-assisted writing systems beyond local editing
suggestions. In addition to producing AI-generated content (AIGC), we also
demonstrate the possibility of using RecurrentGPT as an interactive fiction
that directly interacts with consumers. We call this usage of generative models
by ``AI As Contents'' (AIAC), which we believe is the next form of conventional
AIGC. We further demonstrate the possibility of using RecurrentGPT to create
personalized interactive fiction that directly interacts with readers instead
of interacting with writers. More broadly, RecurrentGPT demonstrates the
utility of borrowing ideas from popular model designs in cognitive science and
deep learning for prompting LLMs. Our code is available at
https://github.com/aiwaves-cn/RecurrentGPT and an online demo is available at
https://www.aiwaves.org/recurrentgpt.
RecurrentGPT combines the long-term memory mechanism of RNNs with large language models to generate arbitrarily long, interpretable text with interactive capabilities.
Transformer RecurrentGPT large language model LLM ChatGPT Long Short-Term Memory LSTM language-based long-short term memory AI-generated content AIGC AI As Contents AIAC interactive fiction