NeuroPrompts: An Adaptive Framework to Optimize Prompts for
  Text-to-Image Generation
Despite impressive recent advances in text-to-image diffusion models,
obtaining high-quality images often requires prompt engineering by humans who
have developed expertise in using them. In this work, we present NeuroPrompts,
an adaptive framework that automatically enhances a user's prompt to improve
the quality of generations produced by text-to-image models. Our framework
utilizes constrained text decoding with a pre-trained language model that has
been adapted to generate prompts similar to those produced by human prompt
engineers. This approach enables higher-quality text-to-image generations and
provides user control over stylistic features via constraint set specification.
We demonstrate the utility of our framework by creating an interactive
application for prompt enhancement and image generation using Stable Diffusion.
Additionally, we conduct experiments utilizing a large dataset of
human-engineered prompts for text-to-image generation and show that our
approach automatically produces enhanced prompts that result in superior image
quality. We make our code, a screencast video demo and a live demo instance of
NeuroPrompts publicly available.
655d7fc5be545cd2456124c4
Shachar Rosenman
admin_assigned
2023-11-22T13:42:49.139Z
64575110cd935d48a4773a38
/avatars/9e5d08965efb316d9c3ffb7eb915aee1.svg
Shachar Rosenman
user
shacharrosenman
655d7fc5be545cd2456124c5
Vasudev Lal
admin_assigned
2023-11-22T13:42:56.900Z
6454021413dba495b2dfebd1
/avatars/16f019f7c321aa24000364a50a1975fe.svg
Vasudev Lal
user
vasudevlal
655d7fc5be545cd2456124c6
Phillip Howard
admin_assigned
2023-11-22T13:43:03.098Z
63605f9388e41d249ecf13c4
/avatars/7721e8039a107647e27cb3f7b4d665a0.svg
Phillip Howard
user
PhillipHoward
NeuroPrompts: An Adaptive Framework to Optimize Prompts for
  Text-to-Image Generation
Despite impressive recent advances in text-to-image diffusion models,
obtaining high-quality images often requires prompt engineering by humans who
have developed expertise in using them. In this work, we present NeuroPrompts,
an adaptive framework that automatically enhances a user's prompt to improve
the quality of generations produced by text-to-image models. Our framework
utilizes constrained text decoding with a pre-trained language model that has
been adapted to generate prompts similar to those produced by human prompt
engineers. This approach enables higher-quality text-to-image generations and
provides user control over stylistic features via constraint set specification.
We demonstrate the utility of our framework by creating an interactive
application for prompt enhancement and image generation using Stable Diffusion.
Additionally, we conduct experiments utilizing a large dataset of
human-engineered prompts for text-to-image generation and show that our
approach automatically produces enhanced prompts that result in superior image
quality. We make our code, a screencast video demo and a live demo instance of
NeuroPrompts publicly available.
An adaptive framework automatically enhances text-to-image prompts to improve generation quality using a pre-trained language model and constrained text decoding.
text-to-image diffusion models prompt engineering adaptive framework constrained text decoding pre-trained language model Stable Diffusion image generation