title: Interactive3D: Create What You Want by Interactive 3D Generation
summary: 3D object generation has undergone significant advancements, yielding
  high-quality results. However, fall short of achieving precise user control,
  often yielding results that do not align with user expectations, thus limiting
  their applicability. User-envisioning 3D object generation faces significant
  challenges in realizing its concepts using current generative models due to
  limited interaction capabilities. Existing methods mainly offer two approaches:
  (i) interpreting textual instructions with constrained controllability, or (ii)
  reconstructing 3D objects from 2D images. Both of them limit customization to
  the confines of the 2D reference and potentially introduce undesirable
  artifacts during the 3D lifting process, restricting the scope for direct and
  versatile 3D modifications. In this work, we introduce Interactive3D, an
  innovative framework for interactive 3D generation that grants users precise
  control over the generative process through extensive 3D interaction
  capabilities. Interactive3D is constructed in two cascading stages, utilizing
  distinct 3D representations. The first stage employs Gaussian Splatting for
  direct user interaction, allowing modifications and guidance of the generative
  direction at any intermediate step through (i) Adding and Removing components,
  (ii) Deformable and Rigid Dragging, (iii) Geometric Transformations, and (iv)
  Semantic Editing. Subsequently, the Gaussian splats are transformed into
  InstantNGP. We introduce a novel (v) Interactive Hash Refinement module to
  further add details and extract the geometry in the second stage. Our
  experiments demonstrate that Interactive3D markedly improves the
  controllability and quality of 3D generation. Our project webpage is available
  at https://interactive-3d.github.io/.
paper_authors._id: 662b088b57b0996d7e778855
paper_authors.name: Shaocong Dong
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-04-26T09:39:34.942Z
paper_authors._id: 63ae91af2314b93f9e6dde42
paper_authors.avatarUrl: /avatars/792ce138cbee85b8754fdcec7fb1ff52.svg
paper_authors.fullname: Shaocong Dong
paper_authors.type: user
paper_authors.user: dscdyc
paper_authors._id: 662b088b57b0996d7e778856
paper_authors.name: Lihe Ding
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-04-26T07:43:28.280Z
paper_authors._id: 64190a6c0956be7233a5f317
paper_authors.avatarUrl: /avatars/8590515089bc31a168bc2da2c7516dfb.svg
paper_authors.fullname: Lihe Ding
paper_authors.type: user
paper_authors.user: Lihe
paper_authors._id: 662b088b57b0996d7e778857
paper_authors.name: Zhanpeng Huang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-04-26T09:39:48.182Z
paper_authors._id: 6351129285bdb764f6aa4c8a
paper_authors.avatarUrl: /avatars/cfc32d19e038caedd65180eb07f87881.svg
paper_authors.fullname: HuangZhanpeng
paper_authors.type: user
paper_authors.user: RyanHZP
paper_authors._id: 662b088b57b0996d7e778858
paper_authors.name: Zibin Wang
paper_authors._id: 662b088b57b0996d7e778859
paper_authors.name: Tianfan Xue
paper_authors._id: 662b088b57b0996d7e77885a
paper_authors.name: Dan Xu
paper_title: Interactive3D: Create What You Want by Interactive 3D Generation
paper_summary: 3D object generation has undergone significant advancements, yielding
  high-quality results. However, fall short of achieving precise user control,
  often yielding results that do not align with user expectations, thus limiting
  their applicability. User-envisioning 3D object generation faces significant
  challenges in realizing its concepts using current generative models due to
  limited interaction capabilities. Existing methods mainly offer two approaches:
  (i) interpreting textual instructions with constrained controllability, or (ii)
  reconstructing 3D objects from 2D images. Both of them limit customization to
  the confines of the 2D reference and potentially introduce undesirable
  artifacts during the 3D lifting process, restricting the scope for direct and
  versatile 3D modifications. In this work, we introduce Interactive3D, an
  innovative framework for interactive 3D generation that grants users precise
  control over the generative process through extensive 3D interaction
  capabilities. Interactive3D is constructed in two cascading stages, utilizing
  distinct 3D representations. The first stage employs Gaussian Splatting for
  direct user interaction, allowing modifications and guidance of the generative
  direction at any intermediate step through (i) Adding and Removing components,
  (ii) Deformable and Rigid Dragging, (iii) Geometric Transformations, and (iv)
  Semantic Editing. Subsequently, the Gaussian splats are transformed into
  InstantNGP. We introduce a novel (v) Interactive Hash Refinement module to
  further add details and extract the geometry in the second stage. Our
  experiments demonstrate that Interactive3D markedly improves the
  controllability and quality of 3D generation. Our project webpage is available
  at https://interactive-3d.github.io/.
paper_ai_summary: Interactive3D framework enhances 3D object generation by allowing precise user control through direct interaction capabilities and iterative refinement.
paper_ai_keywords: Gaussian Splatting Direct user interaction Deformable and Rigid Dragging Geometric Transformations Semantic Editing InstantNGP Interactive Hash Refinement