VisualCloze: A Universal Image Generation Framework via Visual
  In-Context Learning
Recent progress in diffusion models significantly advances various image
generation tasks. However, the current mainstream approach remains focused on
building task-specific models, which have limited efficiency when supporting a
wide range of different needs. While universal models attempt to address this
limitation, they face critical challenges, including generalizable task
instruction, appropriate task distributions, and unified architectural design.
To tackle these challenges, we propose VisualCloze, a universal image
generation framework, which supports a wide range of in-domain tasks,
generalization to unseen ones, unseen unification of multiple tasks, and
reverse generation. Unlike existing methods that rely on language-based task
instruction, leading to task ambiguity and weak generalization, we integrate
visual in-context learning, allowing models to identify tasks from visual
demonstrations. Meanwhile, the inherent sparsity of visual task distributions
hampers the learning of transferable knowledge across tasks. To this end, we
introduce Graph200K, a graph-structured dataset that establishes various
interrelated tasks, enhancing task density and transferable knowledge.
Furthermore, we uncover that our unified image generation formulation shared a
consistent objective with image infilling, enabling us to leverage the strong
generative priors of pre-trained infilling models without modifying the
architectures.
67f86e6884277ab58c40ce8a
Zhong-Yu Li
claimed_verified
2025-04-11T07:25:42.946Z
6740a5730bb4a675446a80ad
/avatars/27c08e33df88e4f73c136da65f2b5adb.svg
Zhong-Yu Li
user
lzyhha
67f86e6884277ab58c40ce8b
Ruoyi Du
admin_assigned
2025-04-11T08:27:33.519Z
64a54586c0f13de8e7093314
/avatars/389e43e9a32cf2fc95f8f3a23b8f0508.svg
Ruoyi Du
user
RuoyiDu
67f86e6884277ab58c40ce8c
Juncheng Yan
admin_assigned
2025-04-11T08:27:41.530Z
644616965691ca69b0e02e79
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/F2uQXO9SkQwHU6benSwQB.jpeg
Juncheng Yan
user
JonsonYan
67f86e6884277ab58c40ce8d
Le Zhuo
67f86e6884277ab58c40ce8e
Zhen Li
claimed_verified
2025-04-13T19:25:40.503Z
6285a9133ab6642179158944
/avatars/6e10fa07c94141fcdbe0cab02bb731ca.svg
Zhen Li
user
Paper99
67f86e6884277ab58c40ce8f
Peng Gao
admin_assigned
2025-04-11T08:28:33.570Z
67b299cc6f6dc4376d9e6c76
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/UniMpmfOUlyiSOrf47wuT.png
Peng Gao
user
cosumosu25
67f86e6884277ab58c40ce90
Zhanyu Ma
67f86e6884277ab58c40ce91
Ming-Ming Cheng
admin_assigned
2025-04-11T08:28:11.683Z
64e496ae0195913c7fa91c66
/avatars/23f274f0a3b4ef6ded35205df9bfb564.svg
chengmingming
user
mingming8688
VisualCloze: A Universal Image Generation Framework via Visual
  In-Context Learning
Recent progress in diffusion models significantly advances various image
generation tasks. However, the current mainstream approach remains focused on
building task-specific models, which have limited efficiency when supporting a
wide range of different needs. While universal models attempt to address this
limitation, they face critical challenges, including generalizable task
instruction, appropriate task distributions, and unified architectural design.
To tackle these challenges, we propose VisualCloze, a universal image
generation framework, which supports a wide range of in-domain tasks,
generalization to unseen ones, unseen unification of multiple tasks, and
reverse generation. Unlike existing methods that rely on language-based task
instruction, leading to task ambiguity and weak generalization, we integrate
visual in-context learning, allowing models to identify tasks from visual
demonstrations. Meanwhile, the inherent sparsity of visual task distributions
hampers the learning of transferable knowledge across tasks. To this end, we
introduce Graph200K, a graph-structured dataset that establishes various
interrelated tasks, enhancing task density and transferable knowledge.
Furthermore, we uncover that our unified image generation formulation shared a
consistent objective with image infilling, enabling us to leverage the strong
generative priors of pre-trained infilling models without modifying the
architectures.
VisualCloze framework enhances universal image generation through visual in-context learning, Graph200K dataset, and leveraging pre-trained infilling models.
diffusion models VisualCloze universal image generation visual in-context learning Graph200K image infilling pre-trained infilling models