Model Dementia: Generated Data Makes Models Forget
Stable Diffusion revolutionised image creation from descriptive text. GPT-2,
GPT-3(.5) and GPT-4 demonstrated astonishing performance across a variety of
language tasks. ChatGPT introduced such language models to the general public.
It is now clear that large language models (LLMs) are here to stay, and will
bring about drastic change in the whole ecosystem of online text and images. In
this paper we consider what the future might hold. What will happen to GPT-{n}
once LLMs contribute much of the language found online? We find that use of
model-generated content in training causes irreversible defects in the
resulting models, where tails of the original content distribution disappear.
We call this effect model dementia and show that it can occur in Variational
Autoencoders (VAEs), Gaussian Mixture Models (GMMs) and LLMs. We build
theoretical intuition behind the phenomenon and portray its ubiquity amongst
all learned generative models. We demonstrate that it has to be taken seriously
if we are to sustain the benefits of training from large-scale data scraped
from the web. Indeed, the value of data collected about genuine human
interactions with systems will be increasingly valuable in the presence of
content generated by LLMs in data crawled from the Internet.
64756056b68461d5cf7eb717
Ilia Shumailov
claimed_verified
2023-05-30T10:07:58.040Z
6475c2794766357252e69e9f
/avatars/757ed789423113369868e972f21ce559.svg
i
user
iliashum
64756056b68461d5cf7eb718
Zakhar Shumaylov
64756056b68461d5cf7eb719
Yiren Zhao
64756056b68461d5cf7eb71a
Yarin Gal
64756056b68461d5cf7eb71b
Nicolas Papernot
64756056b68461d5cf7eb71c
Ross Anderson
claimed_verified
2023-05-30T10:08:01.131Z
6475c2c9975bc84973211a60
/avatars/efd314f9404c01dd419c5aa563fca8d3.svg
Ross Anderson
user
RossJAnderson
Model Dementia: Generated Data Makes Models Forget
Stable Diffusion revolutionised image creation from descriptive text. GPT-2,
GPT-3(.5) and GPT-4 demonstrated astonishing performance across a variety of
language tasks. ChatGPT introduced such language models to the general public.
It is now clear that large language models (LLMs) are here to stay, and will
bring about drastic change in the whole ecosystem of online text and images. In
this paper we consider what the future might hold. What will happen to GPT-{n}
once LLMs contribute much of the language found online? We find that use of
model-generated content in training causes irreversible defects in the
resulting models, where tails of the original content distribution disappear.
We call this effect model dementia and show that it can occur in Variational
Autoencoders (VAEs), Gaussian Mixture Models (GMMs) and LLMs. We build
theoretical intuition behind the phenomenon and portray its ubiquity amongst
all learned generative models. We demonstrate that it has to be taken seriously
if we are to sustain the benefits of training from large-scale data scraped
from the web. Indeed, the value of data collected about genuine human
interactions with systems will be increasingly valuable in the presence of
content generated by LLMs in data crawled from the Internet.
Model dementia, caused by the use of model-generated content in training, affects the quality of generative models like LLMs, VAEs, and GMMs, highlighting the need for genuine human interaction data.
Variational Autoencoders Gaussian Mixture Models large language models model dementia generative models