title: TIP-Editor: An Accurate 3D Editor Following Both Text-Prompts And
    Image-Prompts
summary: Text-driven 3D scene editing has gained significant attention owing to its
  convenience and user-friendliness. However, existing methods still lack
  accurate control of the specified appearance and location of the editing result
  due to the inherent limitations of the text description. To this end, we
  propose a 3D scene editing framework, TIPEditor, that accepts both text and
  image prompts and a 3D bounding box to specify the editing region. With the
  image prompt, users can conveniently specify the detailed appearance/style of
  the target content in complement to the text description, enabling accurate
  control of the appearance. Specifically, TIP-Editor employs a stepwise 2D
  personalization strategy to better learn the representation of the existing
  scene and the reference image, in which a localization loss is proposed to
  encourage correct object placement as specified by the bounding box.
  Additionally, TIPEditor utilizes explicit and flexible 3D Gaussian splatting as
  the 3D representation to facilitate local editing while keeping the background
  unchanged. Extensive experiments have demonstrated that TIP-Editor conducts
  accurate editing following the text and image prompts in the specified bounding
  box region, consistently outperforming the baselines in editing quality, and
  the alignment to the prompts, qualitatively and quantitatively.
paper_authors._id: 65b711f18bb894c96cc4703f
paper_authors.name: Jingyu Zhuang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-29T11:24:35.612Z
paper_authors._id: 63611112108cc69e687b8ea6
paper_authors.avatarUrl: /avatars/8c96fa784ce747d3c1c4bcb09bccf9c0.svg
paper_authors.fullname: zhuang jingyu
paper_authors.type: user
paper_authors.user: zjy526223908
paper_authors._id: 65b711f18bb894c96cc47040
paper_authors.name: Di Kang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-29T11:24:42.417Z
paper_authors._id: 658065828ecea67c7d4b3b00
paper_authors.avatarUrl: /avatars/cb5803480e015e32fc8b6e66b19e4a8d.svg
paper_authors.fullname: Di Kang
paper_authors.type: user
paper_authors.user: madkang
paper_authors._id: 65b711f18bb894c96cc47041
paper_authors.name: Yan-Pei Cao
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2024-01-29T02:48:20.129Z
paper_authors._id: 638066faf022c8a5803f7eb8
paper_authors.avatarUrl: /avatars/4cfd699c3f6c5461b12b7dc5e3fe183d.svg
paper_authors.fullname: Yanpei Cao
paper_authors.type: user
paper_authors.user: pookiefoof
paper_authors._id: 65b711f18bb894c96cc47042
paper_authors.name: Guanbin Li
paper_authors._id: 65b711f18bb894c96cc47043
paper_authors.name: Liang Lin
paper_authors._id: 65b711f18bb894c96cc47044
paper_authors.name: Ying Shan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-01-29T11:25:08.175Z
paper_authors._id: 63ca3ddc04c979828310bfcb
paper_authors.avatarUrl: /avatars/615e0d8622950b4408b40d550f02a894.svg
paper_authors.fullname: Ying Shan
paper_authors.type: user
paper_authors.user: yshan2u
paper_title: TIP-Editor: An Accurate 3D Editor Following Both Text-Prompts And
    Image-Prompts
paper_summary: Text-driven 3D scene editing has gained significant attention owing to its
  convenience and user-friendliness. However, existing methods still lack
  accurate control of the specified appearance and location of the editing result
  due to the inherent limitations of the text description. To this end, we
  propose a 3D scene editing framework, TIPEditor, that accepts both text and
  image prompts and a 3D bounding box to specify the editing region. With the
  image prompt, users can conveniently specify the detailed appearance/style of
  the target content in complement to the text description, enabling accurate
  control of the appearance. Specifically, TIP-Editor employs a stepwise 2D
  personalization strategy to better learn the representation of the existing
  scene and the reference image, in which a localization loss is proposed to
  encourage correct object placement as specified by the bounding box.
  Additionally, TIPEditor utilizes explicit and flexible 3D Gaussian splatting as
  the 3D representation to facilitate local editing while keeping the background
  unchanged. Extensive experiments have demonstrated that TIP-Editor conducts
  accurate editing following the text and image prompts in the specified bounding
  box region, consistently outperforming the baselines in editing quality, and
  the alignment to the prompts, qualitatively and quantitatively.
paper_ai_keywords: 3D Gaussian splatting 2D personalization strategy localization loss 3D scene editing text-driven editing image prompts 3D bounding box