title: Style-NeRF2NeRF: 3D Style Transfer From Style-Aligned Multi-View Images
summary: We propose a simple yet effective pipeline for stylizing a 3D scene,
  harnessing the power of 2D image diffusion models. Given a NeRF model
  reconstructed from a set of multi-view images, we perform 3D style transfer by
  refining the source NeRF model using stylized images generated by a
  style-aligned image-to-image diffusion model. Given a target style prompt, we
  first generate perceptually similar multi-view images by leveraging a
  depth-conditioned diffusion model with an attention-sharing mechanism. Next,
  based on the stylized multi-view images, we propose to guide the style transfer
  process with the sliced Wasserstein loss based on the feature maps extracted
  from a pre-trained CNN model. Our pipeline consists of decoupled steps,
  allowing users to test various prompt ideas and preview the stylized 3D result
  before proceeding to the NeRF fine-tuning stage. We demonstrate that our method
  can transfer diverse artistic styles to real-world 3D scenes with competitive
  quality.
paper_authors._id: 66790934a845e4470fbee8b8
paper_authors.name: Haruo Fujiwara
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-06-24T07:27:21.476Z
paper_authors._id: 639144760cf6b11c487d755d
paper_authors.avatarUrl: /avatars/51895727de5841fbd8e0760c3fc6723b.svg
paper_authors.fullname: Haruo Fujiwara
paper_authors.type: user
paper_authors.user: haruo666
paper_authors._id: 66790934a845e4470fbee8b9
paper_authors.name: Yusuke Mukuta
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-24T13:57:25.510Z
paper_authors._id: 630590f985c9f507e354f430
paper_authors.avatarUrl: /avatars/1ee0c181134c4ae0710968a67484dff4.svg
paper_authors.fullname: Yusuke Mukuta
paper_authors.type: user
paper_authors.user: mukuta
paper_authors._id: 66790934a845e4470fbee8ba
paper_authors.name: Tatsuya Harada
paper_title: Style-NeRF2NeRF: 3D Style Transfer From Style-Aligned Multi-View Images
paper_summary: We propose a simple yet effective pipeline for stylizing a 3D scene,
  harnessing the power of 2D image diffusion models. Given a NeRF model
  reconstructed from a set of multi-view images, we perform 3D style transfer by
  refining the source NeRF model using stylized images generated by a
  style-aligned image-to-image diffusion model. Given a target style prompt, we
  first generate perceptually similar multi-view images by leveraging a
  depth-conditioned diffusion model with an attention-sharing mechanism. Next,
  based on the stylized multi-view images, we propose to guide the style transfer
  process with the sliced Wasserstein loss based on the feature maps extracted
  from a pre-trained CNN model. Our pipeline consists of decoupled steps,
  allowing users to test various prompt ideas and preview the stylized 3D result
  before proceeding to the NeRF fine-tuning stage. We demonstrate that our method
  can transfer diverse artistic styles to real-world 3D scenes with competitive
  quality.
paper_ai_summary: A pipeline for 3D style transfer uses 2D diffusion models and sliced Wasserstein loss to apply artistic styles to real-world 3D scenes reconstructed from NeRF models.
paper_ai_keywords: 3D style transfer 2D diffusion models NeRF models multi-view images depth-conditioned diffusion model attention-sharing mechanism sliced Wasserstein loss CNN model perceptually similar images