title: Hi3D: Pursuing High-Resolution Image-to-3D Generation with Video
    Diffusion Models
summary: Despite having tremendous progress in image-to-3D generation, existing
  methods still struggle to produce multi-view consistent images with
  high-resolution textures in detail, especially in the paradigm of 2D diffusion
  that lacks 3D awareness. In this work, we present High-resolution Image-to-3D
  model (Hi3D), a new video diffusion based paradigm that redefines a single
  image to multi-view images as 3D-aware sequential image generation (i.e.,
  orbital video generation). This methodology delves into the underlying temporal
  consistency knowledge in video diffusion model that generalizes well to
  geometry consistency across multiple views in 3D generation. Technically, Hi3D
  first empowers the pre-trained video diffusion model with 3D-aware prior
  (camera pose condition), yielding multi-view images with low-resolution texture
  details. A 3D-aware video-to-video refiner is learnt to further scale up the
  multi-view images with high-resolution texture details. Such high-resolution
  multi-view images are further augmented with novel views through 3D Gaussian
  Splatting, which are finally leveraged to obtain high-fidelity meshes via 3D
  reconstruction. Extensive experiments on both novel view synthesis and single
  view reconstruction demonstrate that our Hi3D manages to produce superior
  multi-view consistency images with highly-detailed textures. Source code and
  data are available at https://github.com/yanghb22-fdu/Hi3D-Official.
paper_authors._id: 66e249015293a8105aa793d0
paper_authors.name: Haibo Yang
paper_authors._id: 66e249015293a8105aa793d1
paper_authors.name: Yang Chen
paper_authors._id: 66e249015293a8105aa793d2
paper_authors.name: Yingwei Pan
paper_authors._id: 66e249015293a8105aa793d3
paper_authors.name: Ting Yao
paper_authors._id: 66e249015293a8105aa793d4
paper_authors.name: Zhineng Chen
paper_authors._id: 66e249015293a8105aa793d5
paper_authors.name: Chong-Wah Ngo
paper_authors._id: 66e249015293a8105aa793d6
paper_authors.name: Tao Mei
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-12T08:53:45.376Z
paper_authors._id: 66a8d386fbde2b9cb1120393
paper_authors.avatarUrl: /avatars/ccfb6e72764e547be2b3528713151693.svg
paper_authors.fullname: Tao Mei
paper_authors.type: user
paper_authors.user: GiantBision
paper_title: Hi3D: Pursuing High-Resolution Image-to-3D Generation with Video
    Diffusion Models
paper_summary: Despite having tremendous progress in image-to-3D generation, existing
  methods still struggle to produce multi-view consistent images with
  high-resolution textures in detail, especially in the paradigm of 2D diffusion
  that lacks 3D awareness. In this work, we present High-resolution Image-to-3D
  model (Hi3D), a new video diffusion based paradigm that redefines a single
  image to multi-view images as 3D-aware sequential image generation (i.e.,
  orbital video generation). This methodology delves into the underlying temporal
  consistency knowledge in video diffusion model that generalizes well to
  geometry consistency across multiple views in 3D generation. Technically, Hi3D
  first empowers the pre-trained video diffusion model with 3D-aware prior
  (camera pose condition), yielding multi-view images with low-resolution texture
  details. A 3D-aware video-to-video refiner is learnt to further scale up the
  multi-view images with high-resolution texture details. Such high-resolution
  multi-view images are further augmented with novel views through 3D Gaussian
  Splatting, which are finally leveraged to obtain high-fidelity meshes via 3D
  reconstruction. Extensive experiments on both novel view synthesis and single
  view reconstruction demonstrate that our Hi3D manages to produce superior
  multi-view consistency images with highly-detailed textures. Source code and
  data are available at https://github.com/yanghb22-fdu/Hi3D-Official.
paper_ai_summary: Hi3D is a novel video diffusion model that generates high-resolution, multi-view consistent images with detailed textures using 3D-aware priors and refinement techniques.
paper_ai_keywords: video diffusion multi-view images 3D-aware sequential image generation orbital video generation 3D-aware prior camera pose condition 3D-aware video-to-video refiner high-resolution texture details 3D Gaussian Splatting high-fidelity meshes novel view synthesis single view reconstruction