title: HarmonyView: Harmonizing Consistency and Diversity in One-Image-to-3D
summary: Recent progress in single-image 3D generation highlights the importance of
  multi-view coherency, leveraging 3D priors from large-scale diffusion models
  pretrained on Internet-scale images. However, the aspect of novel-view
  diversity remains underexplored within the research landscape due to the
  ambiguity in converting a 2D image into 3D content, where numerous potential
  shapes can emerge. Here, we aim to address this research gap by simultaneously
  addressing both consistency and diversity. Yet, striking a balance between
  these two aspects poses a considerable challenge due to their inherent
  trade-offs. This work introduces HarmonyView, a simple yet effective diffusion
  sampling technique adept at decomposing two intricate aspects in single-image
  3D generation: consistency and diversity. This approach paves the way for a
  more nuanced exploration of the two critical dimensions within the sampling
  process. Moreover, we propose a new evaluation metric based on CLIP image and
  text encoders to comprehensively assess the diversity of the generated views,
  which closely aligns with human evaluators' judgments. In experiments,
  HarmonyView achieves a harmonious balance, demonstrating a win-win scenario in
  both consistency and diversity.
paper_authors._id: 658b900b5c6fb5d5e321d891
paper_authors.name: Sangmin Woo
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-06-30T20:59:00.659Z
paper_authors._id: 64a3d9f467abe0e3d360151d
paper_authors.avatarUrl: /avatars/d0d8772cfdb6e8571f456ef31c7871f1.svg
paper_authors.fullname: Sangmin Woo
paper_authors.type: user
paper_authors.user: sangminwoo
paper_authors._id: 658b900b5c6fb5d5e321d892
paper_authors.name: Byeongjun Park
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-12-27T11:05:48.777Z
paper_authors._id: 653929a66da48e0d21e65e17
paper_authors.avatarUrl: /avatars/e34ae3411d689b4280ff34c1b680f283.svg
paper_authors.fullname: Byeongjun Park
paper_authors.type: user
paper_authors.user: byeongjun-park
paper_authors._id: 658b900b5c6fb5d5e321d893
paper_authors.name: Hyojun Go
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-01-15T07:42:14.585Z
paper_authors._id: 649f65a4ca03a1a35e3dac14
paper_authors.avatarUrl: /avatars/b0dcd8ad795b1e666ee247b2ac024d53.svg
paper_authors.fullname: Hyojun GO
paper_authors.type: user
paper_authors.user: HJGO
paper_authors._id: 658b900b5c6fb5d5e321d894
paper_authors.name: Jin-Young Kim
paper_authors._id: 658b900b5c6fb5d5e321d895
paper_authors.name: Changick Kim
paper_title: HarmonyView: Harmonizing Consistency and Diversity in One-Image-to-3D
paper_summary: Recent progress in single-image 3D generation highlights the importance of
  multi-view coherency, leveraging 3D priors from large-scale diffusion models
  pretrained on Internet-scale images. However, the aspect of novel-view
  diversity remains underexplored within the research landscape due to the
  ambiguity in converting a 2D image into 3D content, where numerous potential
  shapes can emerge. Here, we aim to address this research gap by simultaneously
  addressing both consistency and diversity. Yet, striking a balance between
  these two aspects poses a considerable challenge due to their inherent
  trade-offs. This work introduces HarmonyView, a simple yet effective diffusion
  sampling technique adept at decomposing two intricate aspects in single-image
  3D generation: consistency and diversity. This approach paves the way for a
  more nuanced exploration of the two critical dimensions within the sampling
  process. Moreover, we propose a new evaluation metric based on CLIP image and
  text encoders to comprehensively assess the diversity of the generated views,
  which closely aligns with human evaluators' judgments. In experiments,
  HarmonyView achieves a harmonious balance, demonstrating a win-win scenario in
  both consistency and diversity.
paper_ai_summary: HarmonyView, a diffusion sampling technique, addresses consistency and diversity in single-image 3D generation and introduces a CLIP-based evaluation metric for diversity.
paper_ai_keywords: multi-view coherency diffusion models novel-view diversity single-image 3D generation consistency diversity diffusion sampling technique CLIP image and text encoders