Interactive3D: Create What You Want by Interactive 3D Generation
3D object generation has undergone significant advancements, yielding
high-quality results. However, fall short of achieving precise user control,
often yielding results that do not align with user expectations, thus limiting
their applicability. User-envisioning 3D object generation faces significant
challenges in realizing its concepts using current generative models due to
limited interaction capabilities. Existing methods mainly offer two approaches:
(i) interpreting textual instructions with constrained controllability, or (ii)
reconstructing 3D objects from 2D images. Both of them limit customization to
the confines of the 2D reference and potentially introduce undesirable
artifacts during the 3D lifting process, restricting the scope for direct and
versatile 3D modifications. In this work, we introduce Interactive3D, an
innovative framework for interactive 3D generation that grants users precise
control over the generative process through extensive 3D interaction
capabilities. Interactive3D is constructed in two cascading stages, utilizing
distinct 3D representations. The first stage employs Gaussian Splatting for
direct user interaction, allowing modifications and guidance of the generative
direction at any intermediate step through (i) Adding and Removing components,
(ii) Deformable and Rigid Dragging, (iii) Geometric Transformations, and (iv)
Semantic Editing. Subsequently, the Gaussian splats are transformed into
InstantNGP. We introduce a novel (v) Interactive Hash Refinement module to
further add details and extract the geometry in the second stage. Our
experiments demonstrate that Interactive3D markedly improves the
controllability and quality of 3D generation. Our project webpage is available
at https://interactive-3d.github.io/.
662b088b57b0996d7e778855
Shaocong Dong
admin_assigned
2024-04-26T09:39:34.942Z
63ae91af2314b93f9e6dde42
/avatars/792ce138cbee85b8754fdcec7fb1ff52.svg
Shaocong Dong
user
dscdyc
662b088b57b0996d7e778856
Lihe Ding
claimed_verified
2024-04-26T07:43:28.280Z
64190a6c0956be7233a5f317
/avatars/8590515089bc31a168bc2da2c7516dfb.svg
Lihe Ding
user
Lihe
662b088b57b0996d7e778857
Zhanpeng Huang
admin_assigned
2024-04-26T09:39:48.182Z
6351129285bdb764f6aa4c8a
/avatars/cfc32d19e038caedd65180eb07f87881.svg
HuangZhanpeng
user
RyanHZP
662b088b57b0996d7e778858
Zibin Wang
662b088b57b0996d7e778859
Tianfan Xue
662b088b57b0996d7e77885a
Dan Xu
claimed_verified
2025-07-15T19:15:14.514Z
66feab48651e00e22f33222e
/avatars/7344377e2c796c7ec85194bb2fc78521.svg
Dan Xu
user
danxuhk
Interactive3D: Create What You Want by Interactive 3D Generation
3D object generation has undergone significant advancements, yielding
high-quality results. However, fall short of achieving precise user control,
often yielding results that do not align with user expectations, thus limiting
their applicability. User-envisioning 3D object generation faces significant
challenges in realizing its concepts using current generative models due to
limited interaction capabilities. Existing methods mainly offer two approaches:
(i) interpreting textual instructions with constrained controllability, or (ii)
reconstructing 3D objects from 2D images. Both of them limit customization to
the confines of the 2D reference and potentially introduce undesirable
artifacts during the 3D lifting process, restricting the scope for direct and
versatile 3D modifications. In this work, we introduce Interactive3D, an
innovative framework for interactive 3D generation that grants users precise
control over the generative process through extensive 3D interaction
capabilities. Interactive3D is constructed in two cascading stages, utilizing
distinct 3D representations. The first stage employs Gaussian Splatting for
direct user interaction, allowing modifications and guidance of the generative
direction at any intermediate step through (i) Adding and Removing components,
(ii) Deformable and Rigid Dragging, (iii) Geometric Transformations, and (iv)
Semantic Editing. Subsequently, the Gaussian splats are transformed into
InstantNGP. We introduce a novel (v) Interactive Hash Refinement module to
further add details and extract the geometry in the second stage. Our
experiments demonstrate that Interactive3D markedly improves the
controllability and quality of 3D generation. Our project webpage is available
at https://interactive-3d.github.io/.
Interactive3D framework enhances 3D object generation by allowing precise user control through direct interaction capabilities and iterative refinement.
Gaussian Splatting Direct user interaction Deformable and Rigid Dragging Geometric Transformations Semantic Editing InstantNGP Interactive Hash Refinement