OmniGen: Unified Image Generation
In this work, we introduce OmniGen, a new diffusion model for unified image
generation. Unlike popular diffusion models (e.g., Stable Diffusion), OmniGen
no longer requires additional modules such as ControlNet or IP-Adapter to
process diverse control conditions. OmniGenis characterized by the following
features: 1) Unification: OmniGen not only demonstrates text-to-image
generation capabilities but also inherently supports other downstream tasks,
such as image editing, subject-driven generation, and visual-conditional
generation. Additionally, OmniGen can handle classical computer vision tasks by
transforming them into image generation tasks, such as edge detection and human
pose recognition. 2) Simplicity: The architecture of OmniGen is highly
simplified, eliminating the need for additional text encoders. Moreover, it is
more user-friendly compared to existing diffusion models, enabling complex
tasks to be accomplished through instructions without the need for extra
preprocessing steps (e.g., human pose estimation), thereby significantly
simplifying the workflow of image generation. 3) Knowledge Transfer: Through
learning in a unified format, OmniGen effectively transfers knowledge across
different tasks, manages unseen tasks and domains, and exhibits novel
capabilities. We also explore the model's reasoning capabilities and potential
applications of chain-of-thought mechanism. This work represents the first
attempt at a general-purpose image generation model, and there remain several
unresolved issues. We will open-source the related resources at
https://github.com/VectorSpaceLab/OmniGen to foster advancements in this field.
66ea3b25353c1b9b84254825
Shitao Xiao
claimed_verified
2024-09-18T07:01:29.215Z
62612679bbcbd1c34f1638af
/avatars/c0675d05a52192ee14e9ab1633353956.svg
Xiao
user
Shitao
66ea3b25353c1b9b84254826
Yueze Wang
claimed_verified
2024-09-27T06:58:20.470Z
6458b59c7a7e192202df8fa0
/avatars/33ee716477e5686da8723d01e199cd27.svg
Yueze Wang
user
yzwang
66ea3b25353c1b9b84254827
Junjie Zhou
claimed_verified
2024-09-18T07:01:31.041Z
6564a2ceedae9c33b7654a1f
/avatars/42f09356a1282896573ccb44830cd327.svg
JUNJIE ZHOU
user
JUNJIE99
66ea3b25353c1b9b84254828
Huaying Yuan
admin_assigned
2024-09-18T07:12:24.040Z
66d916a7b86f0d569aa19b60
/avatars/2537cee66afecc2d999e05b01c78d319.svg
huaying Yuan
user
avery00
66ea3b25353c1b9b84254829
Xingrun Xing
claimed_verified
2024-09-20T11:42:53.750Z
651c4e0bc0247c08a46ab2a6
/avatars/3396a34ffb400f576371afc8a5064783.svg
xxr
user
xrxing
66ea3b25353c1b9b8425482a
Ruiran Yan
admin_assigned
2024-09-18T07:12:36.909Z
661ac5b53d7248a6f20080c1
/avatars/26aef5944759c2e4366a71eb8c7fc50a.svg
Ruiran Yan
user
Ruiran
66ea3b25353c1b9b8425482b
Shuting Wang
admin_assigned
2024-09-18T07:12:43.024Z
656d68699dcedd16d5596d26
/avatars/3486af06cc2c1562e09b04bb03360912.svg
Shu-Ting Wang
user
stingw
66ea3b25353c1b9b8425482c
Tiejun Huang
66ea3b25353c1b9b8425482d
Zheng Liu
extracted_pending
2024-09-18T02:30:01.852Z
64a38c590111d5ff6c3d5f2b
/avatars/ef13dc7ce243819bc0da9b04e778b432.svg
zhengliu
user
zl101
OmniGen: Unified Image Generation
In this work, we introduce OmniGen, a new diffusion model for unified image
generation. Unlike popular diffusion models (e.g., Stable Diffusion), OmniGen
no longer requires additional modules such as ControlNet or IP-Adapter to
process diverse control conditions. OmniGenis characterized by the following
features: 1) Unification: OmniGen not only demonstrates text-to-image
generation capabilities but also inherently supports other downstream tasks,
such as image editing, subject-driven generation, and visual-conditional
generation. Additionally, OmniGen can handle classical computer vision tasks by
transforming them into image generation tasks, such as edge detection and human
pose recognition. 2) Simplicity: The architecture of OmniGen is highly
simplified, eliminating the need for additional text encoders. Moreover, it is
more user-friendly compared to existing diffusion models, enabling complex
tasks to be accomplished through instructions without the need for extra
preprocessing steps (e.g., human pose estimation), thereby significantly
simplifying the workflow of image generation. 3) Knowledge Transfer: Through
learning in a unified format, OmniGen effectively transfers knowledge across
different tasks, manages unseen tasks and domains, and exhibits novel
capabilities. We also explore the model's reasoning capabilities and potential
applications of chain-of-thought mechanism. This work represents the first
attempt at a general-purpose image generation model, and there remain several
unresolved issues. We will open-source the related resources at
https://github.com/VectorSpaceLab/OmniGen to foster advancements in this field.
OmniGen is a unified diffusion model for image generation that supports diverse tasks without additional modules, emphasizing simplicity, knowledge transfer, and reasoning capabilities.
diffusion model OmniGen ControlNet IP-Adapter text-to-image generation image editing subject-driven generation visual-conditional generation edge detection human pose recognition text encoders chain-of-thought mechanism