title: ComfyGen: Prompt-Adaptive Workflows for Text-to-Image Generation
summary: The practical use of text-to-image generation has evolved from simple,
  monolithic models to complex workflows that combine multiple specialized
  components. While workflow-based approaches can lead to improved image quality,
  crafting effective workflows requires significant expertise, owing to the large
  number of available components, their complex inter-dependence, and their
  dependence on the generation prompt. Here, we introduce the novel task of
  prompt-adaptive workflow generation, where the goal is to automatically tailor
  a workflow to each user prompt. We propose two LLM-based approaches to tackle
  this task: a tuning-based method that learns from user-preference data, and a
  training-free method that uses the LLM to select existing flows. Both
  approaches lead to improved image quality when compared to monolithic models or
  generic, prompt-independent workflows. Our work shows that prompt-dependent
  flow prediction offers a new pathway to improving text-to-image generation
  quality, complementing existing research directions in the field.
paper_authors._id: 66fe0d6064acd6a2de1864b8
paper_authors.name: Rinon Gal
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-03T09:14:20.320Z
paper_authors._id: 627c1360f19c5eb46d55ba05
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1652712871747-627c1360f19c5eb46d55ba05.jpeg
paper_authors.fullname: Rinon Gal
paper_authors.type: user
paper_authors.user: rinong
paper_authors._id: 66fe0d6064acd6a2de1864b9
paper_authors.name: Adi Haviv
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-03T08:52:49.297Z
paper_authors._id: 610b89ccf5098560ee5fa698
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1628156505064-610b89ccf5098560ee5fa698.jpeg
paper_authors.fullname: Adi Haviv
paper_authors.type: user
paper_authors.user: adihaviv
paper_authors._id: 66fe0d6064acd6a2de1864ba
paper_authors.name: Yuval Alaluf
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-03T08:52:41.386Z
paper_authors._id: 6310c1e0e87051f3e3e3bcc1
paper_authors.avatarUrl: /avatars/61b24a2eace7da0e27fbc4369a046051.svg
paper_authors.fullname: Yuval Alaluf
paper_authors.type: user
paper_authors.user: yuvalalaluf
paper_authors._id: 66fe0d6064acd6a2de1864bb
paper_authors.name: Amit H. Bermano
paper_authors._id: 66fe0d6064acd6a2de1864bc
paper_authors.name: Daniel Cohen-Or
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-03T08:52:28.974Z
paper_authors._id: 628507161949ebcae8e24ec3
paper_authors.avatarUrl: /avatars/008ecb3daa4c8187b5f339f1176b3c39.svg
paper_authors.fullname: Daniel Cohen-Or
paper_authors.type: user
paper_authors.user: cohenor
paper_authors._id: 66fe0d6064acd6a2de1864bd
paper_authors.name: Gal Chechik
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-03T08:52:23.222Z
paper_authors._id: 6493393f357b252af72196c5
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6493393f357b252af72196c5/EWSy18XRcMRa_4XMM3Fu-.jpeg
paper_authors.fullname: Gal Chechik
paper_authors.type: user
paper_authors.user: galchechik
paper_title: ComfyGen: Prompt-Adaptive Workflows for Text-to-Image Generation
paper_summary: The practical use of text-to-image generation has evolved from simple,
  monolithic models to complex workflows that combine multiple specialized
  components. While workflow-based approaches can lead to improved image quality,
  crafting effective workflows requires significant expertise, owing to the large
  number of available components, their complex inter-dependence, and their
  dependence on the generation prompt. Here, we introduce the novel task of
  prompt-adaptive workflow generation, where the goal is to automatically tailor
  a workflow to each user prompt. We propose two LLM-based approaches to tackle
  this task: a tuning-based method that learns from user-preference data, and a
  training-free method that uses the LLM to select existing flows. Both
  approaches lead to improved image quality when compared to monolithic models or
  generic, prompt-independent workflows. Our work shows that prompt-dependent
  flow prediction offers a new pathway to improving text-to-image generation
  quality, complementing existing research directions in the field.
paper_ai_summary: Automatic generation of text-to-image workflows based on user prompts enhances image quality compared to monolithic models or generic workflows.
paper_ai_keywords: prompt-adaptive workflow generation LLM-based approaches tuning-based method training-free method prompt-dependent flow prediction