DI-PCG: Diffusion-based Efficient Inverse Procedural Content Generation
  for High-quality 3D Asset Creation
Procedural Content Generation (PCG) is powerful in creating high-quality 3D
contents, yet controlling it to produce desired shapes is difficult and often
requires extensive parameter tuning. Inverse Procedural Content Generation aims
to automatically find the best parameters under the input condition. However,
existing sampling-based and neural network-based methods still suffer from
numerous sample iterations or limited controllability. In this work, we present
DI-PCG, a novel and efficient method for Inverse PCG from general image
conditions. At its core is a lightweight diffusion transformer model, where PCG
parameters are directly treated as the denoising target and the observed images
as conditions to control parameter generation. DI-PCG is efficient and
effective. With only 7.6M network parameters and 30 GPU hours to train, it
demonstrates superior performance in recovering parameters accurately, and
generalizing well to in-the-wild images. Quantitative and qualitative
experiment results validate the effectiveness of DI-PCG in inverse PCG and
image-to-3D generation tasks. DI-PCG offers a promising approach for efficient
inverse PCG and represents a valuable exploration step towards a 3D generation
path that models how to construct a 3D asset using parametric models.
6764da71bdc5692a8d6bccf6
Wang Zhao
claimed_verified
2024-12-20T08:32:57.404Z
66054d519af76210c2ee4b8e
/avatars/c1854765b1e45ec33602b2cb9443f82a.svg
Wang Zhao
user
thuzhaowang
6764da71bdc5692a8d6bccf7
Yan-Pei Cao
admin_assigned
2024-12-20T09:27:44.668Z
638066faf022c8a5803f7eb8
/avatars/4cfd699c3f6c5461b12b7dc5e3fe183d.svg
Yanpei Cao
user
pookiefoof
6764da71bdc5692a8d6bccf8
Jiale Xu
admin_assigned
2024-12-20T09:28:27.765Z
62c695829db11473f08af1cd
/avatars/cacb54077892a44aef81454dc107df4f.svg
Jiale Xu
user
bluestyle97
6764da71bdc5692a8d6bccf9
Yuejiang Dong
6764da71bdc5692a8d6bccfa
Ying Shan
admin_assigned
2024-12-20T09:28:06.210Z
63ca3ddc04c979828310bfcb
/avatars/615e0d8622950b4408b40d550f02a894.svg
Ying Shan
user
yshan2u
DI-PCG: Diffusion-based Efficient Inverse Procedural Content Generation
  for High-quality 3D Asset Creation
Procedural Content Generation (PCG) is powerful in creating high-quality 3D
contents, yet controlling it to produce desired shapes is difficult and often
requires extensive parameter tuning. Inverse Procedural Content Generation aims
to automatically find the best parameters under the input condition. However,
existing sampling-based and neural network-based methods still suffer from
numerous sample iterations or limited controllability. In this work, we present
DI-PCG, a novel and efficient method for Inverse PCG from general image
conditions. At its core is a lightweight diffusion transformer model, where PCG
parameters are directly treated as the denoising target and the observed images
as conditions to control parameter generation. DI-PCG is efficient and
effective. With only 7.6M network parameters and 30 GPU hours to train, it
demonstrates superior performance in recovering parameters accurately, and
generalizing well to in-the-wild images. Quantitative and qualitative
experiment results validate the effectiveness of DI-PCG in inverse PCG and
image-to-3D generation tasks. DI-PCG offers a promising approach for efficient
inverse PCG and represents a valuable exploration step towards a 3D generation
path that models how to construct a 3D asset using parametric models.
DI-PCG, a lightweight diffusion transformer model, efficiently recovers PCG parameters from images, offering superior performance in inverse PCG and image-to-3D generation.
diffusion transformer model denoising target inverse procedural content generation image-to-3D generation