How Many Van Goghs Does It Take to Van Gogh? Finding the Imitation
  Threshold
Text-to-image models are trained using large datasets collected by scraping
image-text pairs from the internet. These datasets often include private,
copyrighted, and licensed material. Training models on such datasets enables
them to generate images with such content, which might violate copyright laws
and individual privacy. This phenomenon is termed imitation -- generation of
images with content that has recognizable similarity to its training images. In
this work we study the relationship between a concept's frequency in the
training dataset and the ability of a model to imitate it. We seek to determine
the point at which a model was trained on enough instances to imitate a concept
-- the imitation threshold. We posit this question as a new problem: Finding
the Imitation Threshold (FIT) and propose an efficient approach that estimates
the imitation threshold without incurring the colossal cost of training
multiple models from scratch. We experiment with two domains -- human faces and
art styles -- for which we create four datasets, and evaluate three
text-to-image models which were trained on two pretraining datasets. Our
results reveal that the imitation threshold of these models is in the range of
200-600 images, depending on the domain and the model. The imitation threshold
can provide an empirical basis for copyright violation claims and acts as a
guiding principle for text-to-image model developers that aim to comply with
copyright and privacy laws. We release the code and data at
https://github.com/vsahil/MIMETIC-2.git and the project's website is
hosted at https://how-many-van-goghs-does-it-take.github.io.
6717dbde394c8723dfe98018
Sahil Verma
claimed_verified
2024-10-23T07:33:22.758Z
63ba371eefe99543b35701ab
/avatars/87e700af5b31caa4a2680701a897fbd9.svg
Sahil Verma
user
vsahil
6717dbde394c8723dfe98019
Royi Rassin
6717dbde394c8723dfe9801a
Arnav Das
6717dbde394c8723dfe9801b
Gantavya Bhatt
6717dbde394c8723dfe9801c
Preethi Seshadri
6717dbde394c8723dfe9801d
Chirag Shah
claimed_verified
2025-03-17T08:46:07.107Z
6624485b0e31d65ecce09748
/avatars/b78085aa72e673e570843b85b116f8bc.svg
Chirag Shah
user
chiragshah
6717dbde394c8723dfe9801e
Jeff Bilmes
6717dbde394c8723dfe9801f
Hannaneh Hajishirzi
6717dbde394c8723dfe98020
Yanai Elazar
How Many Van Goghs Does It Take to Van Gogh? Finding the Imitation
  Threshold
Text-to-image models are trained using large datasets collected by scraping
image-text pairs from the internet. These datasets often include private,
copyrighted, and licensed material. Training models on such datasets enables
them to generate images with such content, which might violate copyright laws
and individual privacy. This phenomenon is termed imitation -- generation of
images with content that has recognizable similarity to its training images. In
this work we study the relationship between a concept's frequency in the
training dataset and the ability of a model to imitate it. We seek to determine
the point at which a model was trained on enough instances to imitate a concept
-- the imitation threshold. We posit this question as a new problem: Finding
the Imitation Threshold (FIT) and propose an efficient approach that estimates
the imitation threshold without incurring the colossal cost of training
multiple models from scratch. We experiment with two domains -- human faces and
art styles -- for which we create four datasets, and evaluate three
text-to-image models which were trained on two pretraining datasets. Our
results reveal that the imitation threshold of these models is in the range of
200-600 images, depending on the domain and the model. The imitation threshold
can provide an empirical basis for copyright violation claims and acts as a
guiding principle for text-to-image model developers that aim to comply with
copyright and privacy laws. We release the code and data at
https://github.com/vsahil/MIMETIC-2.git and the project's website is
hosted at https://how-many-van-goghs-does-it-take.github.io.
Research investigates the relationship between training dataset size and the ability of text-to-image models to imitate concepts, proposing a method to estimate the imitation threshold for copyright compliance.
text-to-image models imitation imitation threshold FIT training datasets human faces art styles pretraining datasets copyright violation privacy laws