title: Collaborative Score Distillation for Consistent Visual Synthesis
summary: Generative priors of large-scale text-to-image diffusion models enable a wide
  range of new generation and editing applications on diverse visual modalities.
  However, when adapting these priors to complex visual modalities, often
  represented as multiple images (e.g., video), achieving consistency across a
  set of images is challenging. In this paper, we address this challenge with a
  novel method, Collaborative Score Distillation (CSD). CSD is based on the Stein
  Variational Gradient Descent (SVGD). Specifically, we propose to consider
  multiple samples as "particles" in the SVGD update and combine their score
  functions to distill generative priors over a set of images synchronously.
  Thus, CSD facilitates seamless integration of information across 2D images,
  leading to a consistent visual synthesis across multiple samples. We show the
  effectiveness of CSD in a variety of tasks, encompassing the visual editing of
  panorama images, videos, and 3D scenes. Our results underline the competency of
  CSD as a versatile method for enhancing inter-sample consistency, thereby
  broadening the applicability of text-to-image diffusion models.
paper_authors._id: 64ae040b6864362a7e2311ce
paper_authors.name: Subin Kim
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-07-12T13:28:48.280Z
paper_authors._id: 63eb83b74dcaaf08763766a9
paper_authors.avatarUrl: /avatars/11e3e00c49a3b74947cce2f1ef624944.svg
paper_authors.fullname: Subin Kim
paper_authors.type: user
paper_authors.user: subin-kim
paper_authors._id: 64ae040b6864362a7e2311cf
paper_authors.name: Kyungmin Lee
paper_authors._id: 64ae040b6864362a7e2311d0
paper_authors.name: June Suk Choi
paper_authors._id: 64ae040b6864362a7e2311d1
paper_authors.name: Jongheon Jeong
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-07-13T08:32:47.469Z
paper_authors._id: 630d92f54ca0a22768c2e76a
paper_authors.avatarUrl: /avatars/92d613ddfc79cd44cc0c808bb3782282.svg
paper_authors.fullname: Jeong
paper_authors.type: user
paper_authors.user: Jongheon
paper_authors._id: 64ae040b6864362a7e2311d2
paper_authors.name: Kihyuk Sohn
paper_authors._id: 64ae040b6864362a7e2311d3
paper_authors.name: Jinwoo Shin
paper_title: Collaborative Score Distillation for Consistent Visual Synthesis
paper_summary: Generative priors of large-scale text-to-image diffusion models enable a wide
  range of new generation and editing applications on diverse visual modalities.
  However, when adapting these priors to complex visual modalities, often
  represented as multiple images (e.g., video), achieving consistency across a
  set of images is challenging. In this paper, we address this challenge with a
  novel method, Collaborative Score Distillation (CSD). CSD is based on the Stein
  Variational Gradient Descent (SVGD). Specifically, we propose to consider
  multiple samples as "particles" in the SVGD update and combine their score
  functions to distill generative priors over a set of images synchronously.
  Thus, CSD facilitates seamless integration of information across 2D images,
  leading to a consistent visual synthesis across multiple samples. We show the
  effectiveness of CSD in a variety of tasks, encompassing the visual editing of
  panorama images, videos, and 3D scenes. Our results underline the competency of
  CSD as a versatile method for enhancing inter-sample consistency, thereby
  broadening the applicability of text-to-image diffusion models.
paper_ai_keywords: generative priors text-to-image diffusion models Stein Variational Gradient Descent (SVGD) particles score functions score distillation visual synthesis panorama images inter-sample consistency