title: OmniGen: Unified Image Generation
summary: In this work, we introduce OmniGen, a new diffusion model for unified image
  generation. Unlike popular diffusion models (e.g., Stable Diffusion), OmniGen
  no longer requires additional modules such as ControlNet or IP-Adapter to
  process diverse control conditions. OmniGenis characterized by the following
  features: 1) Unification: OmniGen not only demonstrates text-to-image
  generation capabilities but also inherently supports other downstream tasks,
  such as image editing, subject-driven generation, and visual-conditional
  generation. Additionally, OmniGen can handle classical computer vision tasks by
  transforming them into image generation tasks, such as edge detection and human
  pose recognition. 2) Simplicity: The architecture of OmniGen is highly
  simplified, eliminating the need for additional text encoders. Moreover, it is
  more user-friendly compared to existing diffusion models, enabling complex
  tasks to be accomplished through instructions without the need for extra
  preprocessing steps (e.g., human pose estimation), thereby significantly
  simplifying the workflow of image generation. 3) Knowledge Transfer: Through
  learning in a unified format, OmniGen effectively transfers knowledge across
  different tasks, manages unseen tasks and domains, and exhibits novel
  capabilities. We also explore the model's reasoning capabilities and potential
  applications of chain-of-thought mechanism. This work represents the first
  attempt at a general-purpose image generation model, and there remain several
  unresolved issues. We will open-source the related resources at
  https://github.com/VectorSpaceLab/OmniGen to foster advancements in this field.
paper_authors._id: 66ea3b25353c1b9b84254825
paper_authors.name: Shitao Xiao
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-09-18T07:01:29.215Z
paper_authors._id: 62612679bbcbd1c34f1638af
paper_authors.avatarUrl: /avatars/c0675d05a52192ee14e9ab1633353956.svg
paper_authors.fullname: Xiao
paper_authors.type: user
paper_authors.user: Shitao
paper_authors._id: 66ea3b25353c1b9b84254826
paper_authors.name: Yueze Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-09-27T06:58:20.470Z
paper_authors._id: 6458b59c7a7e192202df8fa0
paper_authors.avatarUrl: /avatars/33ee716477e5686da8723d01e199cd27.svg
paper_authors.fullname: Yueze Wang
paper_authors.type: user
paper_authors.user: yzwang
paper_authors._id: 66ea3b25353c1b9b84254827
paper_authors.name: Junjie Zhou
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-09-18T07:01:31.041Z
paper_authors._id: 6564a2ceedae9c33b7654a1f
paper_authors.avatarUrl: /avatars/42f09356a1282896573ccb44830cd327.svg
paper_authors.fullname: JUNJIE ZHOU
paper_authors.type: user
paper_authors.user: JUNJIE99
paper_authors._id: 66ea3b25353c1b9b84254828
paper_authors.name: Huaying Yuan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-18T07:12:24.040Z
paper_authors._id: 66d916a7b86f0d569aa19b60
paper_authors.avatarUrl: /avatars/2537cee66afecc2d999e05b01c78d319.svg
paper_authors.fullname: huaying Yuan
paper_authors.type: user
paper_authors.user: avery00
paper_authors._id: 66ea3b25353c1b9b84254829
paper_authors.name: Xingrun Xing
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-09-20T11:42:53.750Z
paper_authors._id: 651c4e0bc0247c08a46ab2a6
paper_authors.avatarUrl: /avatars/3396a34ffb400f576371afc8a5064783.svg
paper_authors.fullname: xxr
paper_authors.type: user
paper_authors.user: xrxing
paper_authors._id: 66ea3b25353c1b9b8425482a
paper_authors.name: Ruiran Yan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-18T07:12:36.909Z
paper_authors._id: 661ac5b53d7248a6f20080c1
paper_authors.avatarUrl: /avatars/26aef5944759c2e4366a71eb8c7fc50a.svg
paper_authors.fullname: Ruiran Yan
paper_authors.type: user
paper_authors.user: Ruiran
paper_authors._id: 66ea3b25353c1b9b8425482b
paper_authors.name: Shuting Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-18T07:12:43.024Z
paper_authors._id: 656d68699dcedd16d5596d26
paper_authors.avatarUrl: /avatars/3486af06cc2c1562e09b04bb03360912.svg
paper_authors.fullname: Shu-Ting Wang
paper_authors.type: user
paper_authors.user: stingw
paper_authors._id: 66ea3b25353c1b9b8425482c
paper_authors.name: Tiejun Huang
paper_authors._id: 66ea3b25353c1b9b8425482d
paper_authors.name: Zheng Liu
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2024-09-18T02:30:01.852Z
paper_authors._id: 64a38c590111d5ff6c3d5f2b
paper_authors.avatarUrl: /avatars/ef13dc7ce243819bc0da9b04e778b432.svg
paper_authors.fullname: zhengliu
paper_authors.type: user
paper_authors.user: zl101
paper_title: OmniGen: Unified Image Generation
paper_summary: In this work, we introduce OmniGen, a new diffusion model for unified image
  generation. Unlike popular diffusion models (e.g., Stable Diffusion), OmniGen
  no longer requires additional modules such as ControlNet or IP-Adapter to
  process diverse control conditions. OmniGenis characterized by the following
  features: 1) Unification: OmniGen not only demonstrates text-to-image
  generation capabilities but also inherently supports other downstream tasks,
  such as image editing, subject-driven generation, and visual-conditional
  generation. Additionally, OmniGen can handle classical computer vision tasks by
  transforming them into image generation tasks, such as edge detection and human
  pose recognition. 2) Simplicity: The architecture of OmniGen is highly
  simplified, eliminating the need for additional text encoders. Moreover, it is
  more user-friendly compared to existing diffusion models, enabling complex
  tasks to be accomplished through instructions without the need for extra
  preprocessing steps (e.g., human pose estimation), thereby significantly
  simplifying the workflow of image generation. 3) Knowledge Transfer: Through
  learning in a unified format, OmniGen effectively transfers knowledge across
  different tasks, manages unseen tasks and domains, and exhibits novel
  capabilities. We also explore the model's reasoning capabilities and potential
  applications of chain-of-thought mechanism. This work represents the first
  attempt at a general-purpose image generation model, and there remain several
  unresolved issues. We will open-source the related resources at
  https://github.com/VectorSpaceLab/OmniGen to foster advancements in this field.
paper_ai_keywords: OmniGen diffusion model Stable Diffusion ControlNet IP-Adapter text-to-image generation image editing subject-driven generation visual-conditional generation edge detection human pose recognition text encoders chain-of-thought mechanism general-purpose image generation model