title: VisualCloze: A Universal Image Generation Framework via Visual
    In-Context Learning
summary: Recent progress in diffusion models significantly advances various image
  generation tasks. However, the current mainstream approach remains focused on
  building task-specific models, which have limited efficiency when supporting a
  wide range of different needs. While universal models attempt to address this
  limitation, they face critical challenges, including generalizable task
  instruction, appropriate task distributions, and unified architectural design.
  To tackle these challenges, we propose VisualCloze, a universal image
  generation framework, which supports a wide range of in-domain tasks,
  generalization to unseen ones, unseen unification of multiple tasks, and
  reverse generation. Unlike existing methods that rely on language-based task
  instruction, leading to task ambiguity and weak generalization, we integrate
  visual in-context learning, allowing models to identify tasks from visual
  demonstrations. Meanwhile, the inherent sparsity of visual task distributions
  hampers the learning of transferable knowledge across tasks. To this end, we
  introduce Graph200K, a graph-structured dataset that establishes various
  interrelated tasks, enhancing task density and transferable knowledge.
  Furthermore, we uncover that our unified image generation formulation shared a
  consistent objective with image infilling, enabling us to leverage the strong
  generative priors of pre-trained infilling models without modifying the
  architectures.
paper_authors._id: 67f86e6884277ab58c40ce8a
paper_authors.name: Zhong-Yu Li
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-11T07:25:42.946Z
paper_authors._id: 6740a5730bb4a675446a80ad
paper_authors.avatarUrl: /avatars/27c08e33df88e4f73c136da65f2b5adb.svg
paper_authors.fullname: Zhong-Yu Li
paper_authors.type: user
paper_authors.user: lzyhha
paper_authors._id: 67f86e6884277ab58c40ce8b
paper_authors.name: Ruoyi Du
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-11T08:27:33.519Z
paper_authors._id: 64a54586c0f13de8e7093314
paper_authors.avatarUrl: /avatars/389e43e9a32cf2fc95f8f3a23b8f0508.svg
paper_authors.fullname: Ruoyi Du
paper_authors.type: user
paper_authors.user: RuoyiDu
paper_authors._id: 67f86e6884277ab58c40ce8c
paper_authors.name: Juncheng Yan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-11T08:27:41.530Z
paper_authors._id: 644616965691ca69b0e02e79
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/F2uQXO9SkQwHU6benSwQB.jpeg
paper_authors.fullname: Juncheng Yan
paper_authors.type: user
paper_authors.user: JonsonYan
paper_authors._id: 67f86e6884277ab58c40ce8d
paper_authors.name: Le Zhuo
paper_authors._id: 67f86e6884277ab58c40ce8e
paper_authors.name: Zhen Li
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-13T19:25:40.503Z
paper_authors._id: 6285a9133ab6642179158944
paper_authors.avatarUrl: /avatars/6e10fa07c94141fcdbe0cab02bb731ca.svg
paper_authors.fullname: Zhen Li
paper_authors.type: user
paper_authors.user: Paper99
paper_authors._id: 67f86e6884277ab58c40ce8f
paper_authors.name: Peng Gao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-11T08:28:33.570Z
paper_authors._id: 67b299cc6f6dc4376d9e6c76
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/UniMpmfOUlyiSOrf47wuT.png
paper_authors.fullname: Peng Gao
paper_authors.type: user
paper_authors.user: cosumosu25
paper_authors._id: 67f86e6884277ab58c40ce90
paper_authors.name: Zhanyu Ma
paper_authors._id: 67f86e6884277ab58c40ce91
paper_authors.name: Ming-Ming Cheng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-11T08:28:11.683Z
paper_authors._id: 64e496ae0195913c7fa91c66
paper_authors.avatarUrl: /avatars/23f274f0a3b4ef6ded35205df9bfb564.svg
paper_authors.fullname: chengmingming
paper_authors.type: user
paper_authors.user: mingming8688
paper_title: VisualCloze: A Universal Image Generation Framework via Visual
    In-Context Learning
paper_summary: Recent progress in diffusion models significantly advances various image
  generation tasks. However, the current mainstream approach remains focused on
  building task-specific models, which have limited efficiency when supporting a
  wide range of different needs. While universal models attempt to address this
  limitation, they face critical challenges, including generalizable task
  instruction, appropriate task distributions, and unified architectural design.
  To tackle these challenges, we propose VisualCloze, a universal image
  generation framework, which supports a wide range of in-domain tasks,
  generalization to unseen ones, unseen unification of multiple tasks, and
  reverse generation. Unlike existing methods that rely on language-based task
  instruction, leading to task ambiguity and weak generalization, we integrate
  visual in-context learning, allowing models to identify tasks from visual
  demonstrations. Meanwhile, the inherent sparsity of visual task distributions
  hampers the learning of transferable knowledge across tasks. To this end, we
  introduce Graph200K, a graph-structured dataset that establishes various
  interrelated tasks, enhancing task density and transferable knowledge.
  Furthermore, we uncover that our unified image generation formulation shared a
  consistent objective with image infilling, enabling us to leverage the strong
  generative priors of pre-trained infilling models without modifying the
  architectures.
paper_ai_summary: VisualCloze framework enhances universal image generation through visual in-context learning, Graph200K dataset, and leveraging pre-trained infilling models.
paper_ai_keywords: diffusion models VisualCloze universal image generation visual in-context learning Graph200K image infilling pre-trained infilling models