title: Context Diffusion: In-Context Aware Image Generation
summary: We propose Context Diffusion, a diffusion-based framework that enables image
  generation models to learn from visual examples presented in context. Recent
  work tackles such in-context learning for image generation, where a query image
  is provided alongside context examples and text prompts. However, the quality
  and fidelity of the generated images deteriorate when the prompt is not
  present, demonstrating that these models are unable to truly learn from the
  visual context. To address this, we propose a novel framework that separates
  the encoding of the visual context and preserving the structure of the query
  images. This results in the ability to learn from the visual context and text
  prompts, but also from either one of them. Furthermore, we enable our model to
  handle few-shot settings, to effectively address diverse in-context learning
  scenarios. Our experiments and user study demonstrate that Context Diffusion
  excels in both in-domain and out-of-domain tasks, resulting in an overall
  enhancement in image quality and fidelity compared to counterpart models.
paper_authors._id: 657131be4c829fb8f5e415eb
paper_authors.name: Ivona Najdenkoska
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-12-07T15:29:52.430Z
paper_authors._id: 6571e0ab487dfeeb4171ca4d
paper_authors.avatarUrl: /avatars/2ab286ad1f502252d71bfed8a54f4cbf.svg
paper_authors.fullname: Ivona Najdenkoska
paper_authors.type: user
paper_authors.user: ivonajdenkoska
paper_authors._id: 657131be4c829fb8f5e415ec
paper_authors.name: Animesh Sinha
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-07T12:05:19.952Z
paper_authors._id: 63772d28e069bdcdbb8e4a29
paper_authors.avatarUrl: /avatars/be71b00e009f2093a3ff79f07815e68e.svg
paper_authors.fullname: Animesh Sinha
paper_authors.type: user
paper_authors.user: animeshsinha
paper_authors._id: 657131be4c829fb8f5e415ed
paper_authors.name: Abhimanyu Dubey
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-07T12:05:27.380Z
paper_authors._id: 6376f5a98272ecd8ddacacf8
paper_authors.avatarUrl: /avatars/c57d5de9a90ec12b7f617327b1ac27fd.svg
paper_authors.fullname: Abhimanyu Dubey
paper_authors.type: user
paper_authors.user: dubeya
paper_authors._id: 657131be4c829fb8f5e415ee
paper_authors.name: Dhruv Mahajan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-07T12:05:33.968Z
paper_authors._id: 64358a51f81a16e7436330f7
paper_authors.avatarUrl: /avatars/71c350836e5d025b4c7b9f99f72c2e21.svg
paper_authors.fullname: Dhruv Mahajan
paper_authors.type: user
paper_authors.user: Dhruv2009
paper_authors._id: 657131be4c829fb8f5e415ef
paper_authors.name: Vignesh Ramanathan
paper_authors._id: 657131be4c829fb8f5e415f0
paper_authors.name: Filip Radenovic
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-07T12:05:56.935Z
paper_authors._id: 6515dccbc979eb9ca72dcc8c
paper_authors.avatarUrl: /avatars/b55101dc2b0052c36f0b2cbdea9473e7.svg
paper_authors.fullname: Filip Radenovic
paper_authors.type: user
paper_authors.user: filipradenovic
paper_title: Context Diffusion: In-Context Aware Image Generation
paper_summary: We propose Context Diffusion, a diffusion-based framework that enables image
  generation models to learn from visual examples presented in context. Recent
  work tackles such in-context learning for image generation, where a query image
  is provided alongside context examples and text prompts. However, the quality
  and fidelity of the generated images deteriorate when the prompt is not
  present, demonstrating that these models are unable to truly learn from the
  visual context. To address this, we propose a novel framework that separates
  the encoding of the visual context and preserving the structure of the query
  images. This results in the ability to learn from the visual context and text
  prompts, but also from either one of them. Furthermore, we enable our model to
  handle few-shot settings, to effectively address diverse in-context learning
  scenarios. Our experiments and user study demonstrate that Context Diffusion
  excels in both in-domain and out-of-domain tasks, resulting in an overall
  enhancement in image quality and fidelity compared to counterpart models.
paper_ai_keywords: Context Diffusion diffusion-based framework image generation models visual examples context learning query image context examples text prompts visual context image quality image fidelity few-shot settings in-context learning in-domain tasks out-of-domain tasks counterpart models