Style-NeRF2NeRF: 3D Style Transfer From Style-Aligned Multi-View Images
We propose a simple yet effective pipeline for stylizing a 3D scene,
harnessing the power of 2D image diffusion models. Given a NeRF model
reconstructed from a set of multi-view images, we perform 3D style transfer by
refining the source NeRF model using stylized images generated by a
style-aligned image-to-image diffusion model. Given a target style prompt, we
first generate perceptually similar multi-view images by leveraging a
depth-conditioned diffusion model with an attention-sharing mechanism. Next,
based on the stylized multi-view images, we propose to guide the style transfer
process with the sliced Wasserstein loss based on the feature maps extracted
from a pre-trained CNN model. Our pipeline consists of decoupled steps,
allowing users to test various prompt ideas and preview the stylized 3D result
before proceeding to the NeRF fine-tuning stage. We demonstrate that our method
can transfer diverse artistic styles to real-world 3D scenes with competitive
quality.
66790934a845e4470fbee8b8
Haruo Fujiwara
claimed_verified
2024-06-24T07:27:21.476Z
639144760cf6b11c487d755d
/avatars/51895727de5841fbd8e0760c3fc6723b.svg
Haruo Fujiwara
user
haruo666
66790934a845e4470fbee8b9
Yusuke Mukuta
admin_assigned
2024-06-24T13:57:25.510Z
630590f985c9f507e354f430
/avatars/1ee0c181134c4ae0710968a67484dff4.svg
Yusuke Mukuta
user
mukuta
66790934a845e4470fbee8ba
Tatsuya Harada
Style-NeRF2NeRF: 3D Style Transfer From Style-Aligned Multi-View Images
We propose a simple yet effective pipeline for stylizing a 3D scene,
harnessing the power of 2D image diffusion models. Given a NeRF model
reconstructed from a set of multi-view images, we perform 3D style transfer by
refining the source NeRF model using stylized images generated by a
style-aligned image-to-image diffusion model. Given a target style prompt, we
first generate perceptually similar multi-view images by leveraging a
depth-conditioned diffusion model with an attention-sharing mechanism. Next,
based on the stylized multi-view images, we propose to guide the style transfer
process with the sliced Wasserstein loss based on the feature maps extracted
from a pre-trained CNN model. Our pipeline consists of decoupled steps,
allowing users to test various prompt ideas and preview the stylized 3D result
before proceeding to the NeRF fine-tuning stage. We demonstrate that our method
can transfer diverse artistic styles to real-world 3D scenes with competitive
quality.
A pipeline for 3D style transfer uses 2D diffusion models and sliced Wasserstein loss to apply artistic styles to real-world 3D scenes reconstructed from NeRF models.
3D style transfer 2D diffusion models NeRF models multi-view images depth-conditioned diffusion model attention-sharing mechanism sliced Wasserstein loss CNN model perceptually similar images