CommonCanvas: An Open Diffusion Model Trained with Creative-Commons
  Images
We assemble a dataset of Creative-Commons-licensed (CC) images, which we use
to train a set of open diffusion models that are qualitatively competitive with
Stable Diffusion 2 (SD2). This task presents two challenges: (1)
high-resolution CC images lack the captions necessary to train text-to-image
generative models; (2) CC images are relatively scarce. In turn, to address
these challenges, we use an intuitive transfer learning technique to produce a
set of high-quality synthetic captions paired with curated CC images. We then
develop a data- and compute-efficient training recipe that requires as little
as 3% of the LAION-2B data needed to train existing SD2 models, but obtains
comparable quality. These results indicate that we have a sufficient number of
CC images (~70 million) for training high-quality models. Our training recipe
also implements a variety of optimizations that achieve ~3X training speed-ups,
enabling rapid model iteration. We leverage this recipe to train several
high-quality text-to-image models, which we dub the CommonCanvas family. Our
largest model achieves comparable performance to SD2 on a human evaluation,
despite being trained on our CC dataset that is significantly smaller than
LAION and using synthetic captions for training. We release our models, data,
and code at
https://github.com/mosaicml/diffusion/blob/main/assets/common-canvas.md
6539e114f186c8b4a8b2525c
Aaron Gokaslan
admin_assigned
2023-10-26T08:18:55.314Z
620573d0522e40b4a18d8763
/avatars/9353c064ef8ccac84d0397411d38fa90.svg
Aaron Gokaslan
user
Skylion007
6539e114f186c8b4a8b2525d
A. Feder Cooper
6539e114f186c8b4a8b2525e
Jasmine Collins
admin_assigned
2023-10-26T08:19:12.906Z
64b780de324038715946a2bc
/avatars/4c3f695d7d0232a8d2be70b2ca222910.svg
Jasmine Collins
user
3coepugOsubcba
6539e114f186c8b4a8b2525f
Landan Seguin
claimed_verified
2023-10-27T09:12:46.966Z
63278bbc28396be9bfec8dbb
/avatars/75ff28af26bbee4e243ed33ba4856ddb.svg
Landan Seguin
user
Landan
6539e114f186c8b4a8b25260
Austin Jacobson
claimed_verified
2024-01-05T11:31:54.810Z
630410a1fc783bfc7444aca2
/avatars/d393466560bff8ca45c420a1315b75b6.svg
austin
user
austinj
6539e114f186c8b4a8b25261
Mihir Patel
admin_assigned
2023-10-26T08:20:25.947Z
6453f6f9bc2ca42503ecf4d8
/avatars/988f0264dc8e34aa28cae9b77336b0f1.svg
Mihir Patel
user
mihir-mosaic
6539e114f186c8b4a8b25262
Jonathan Frankle
admin_assigned
2023-10-26T08:20:33.713Z
62b0b7f9a596c50b1783fe9e
https://cdn-avatars.huggingface.co/v1/production/uploads/62b0b7f9a596c50b1783fe9e/nzctJXoyiEwCnOHVJEx3_.png
Jonathan Frankle
user
jfrankle
6539e114f186c8b4a8b25263
Cory Stephenson
admin_assigned
2023-10-26T08:20:40.592Z
638e714750a4e4beef99c802
/avatars/73006dc9b196986511bcc9c5d7cb0fc8.svg
Cory Stephenson
user
CoryMosaicML
6539e114f186c8b4a8b25264
Volodymyr Kuleshov
admin_assigned
2023-10-26T08:20:46.782Z
640ea1c43c82bd463ee80e19
/avatars/3213d2e8a433a207dfb96a35f0f52a92.svg
Volodymyr Kuleshov
user
kuleshov
CommonCanvas: An Open Diffusion Model Trained with Creative-Commons
  Images
We assemble a dataset of Creative-Commons-licensed (CC) images, which we use
to train a set of open diffusion models that are qualitatively competitive with
Stable Diffusion 2 (SD2). This task presents two challenges: (1)
high-resolution CC images lack the captions necessary to train text-to-image
generative models; (2) CC images are relatively scarce. In turn, to address
these challenges, we use an intuitive transfer learning technique to produce a
set of high-quality synthetic captions paired with curated CC images. We then
develop a data- and compute-efficient training recipe that requires as little
as 3% of the LAION-2B data needed to train existing SD2 models, but obtains
comparable quality. These results indicate that we have a sufficient number of
CC images (~70 million) for training high-quality models. Our training recipe
also implements a variety of optimizations that achieve ~3X training speed-ups,
enabling rapid model iteration. We leverage this recipe to train several
high-quality text-to-image models, which we dub the CommonCanvas family. Our
largest model achieves comparable performance to SD2 on a human evaluation,
despite being trained on our CC dataset that is significantly smaller than
LAION and using synthetic captions for training. We release our models, data,
and code at
https://github.com/mosaicml/diffusion/blob/main/assets/common-canvas.md
A transfer learning technique and efficient training recipe using Creative Commons images enable high-quality text-to-image models with synthetic captions and reduced computational requirements.
transfer learning synthetic captions data- and compute-efficient training LAION-2B text-to-image models CommonCanvas family human evaluation