TIP-Editor: An Accurate 3D Editor Following Both Text-Prompts And
  Image-Prompts
Text-driven 3D scene editing has gained significant attention owing to its
convenience and user-friendliness. However, existing methods still lack
accurate control of the specified appearance and location of the editing result
due to the inherent limitations of the text description. To this end, we
propose a 3D scene editing framework, TIPEditor, that accepts both text and
image prompts and a 3D bounding box to specify the editing region. With the
image prompt, users can conveniently specify the detailed appearance/style of
the target content in complement to the text description, enabling accurate
control of the appearance. Specifically, TIP-Editor employs a stepwise 2D
personalization strategy to better learn the representation of the existing
scene and the reference image, in which a localization loss is proposed to
encourage correct object placement as specified by the bounding box.
Additionally, TIPEditor utilizes explicit and flexible 3D Gaussian splatting as
the 3D representation to facilitate local editing while keeping the background
unchanged. Extensive experiments have demonstrated that TIP-Editor conducts
accurate editing following the text and image prompts in the specified bounding
box region, consistently outperforming the baselines in editing quality, and
the alignment to the prompts, qualitatively and quantitatively.
65b711f18bb894c96cc4703f
Jingyu Zhuang
admin_assigned
2024-01-29T11:24:35.612Z
63611112108cc69e687b8ea6
/avatars/8c96fa784ce747d3c1c4bcb09bccf9c0.svg
zhuang jingyu
user
zjy526223908
65b711f18bb894c96cc47040
Di Kang
admin_assigned
2024-01-29T11:24:42.417Z
658065828ecea67c7d4b3b00
/avatars/cb5803480e015e32fc8b6e66b19e4a8d.svg
Di Kang
user
madkang
65b711f18bb894c96cc47041
Yan-Pei Cao
extracted_pending
2024-01-29T02:48:20.129Z
638066faf022c8a5803f7eb8
/avatars/4cfd699c3f6c5461b12b7dc5e3fe183d.svg
Yanpei Cao
user
pookiefoof
65b711f18bb894c96cc47042
Guanbin Li
65b711f18bb894c96cc47043
Liang Lin
65b711f18bb894c96cc47044
Ying Shan
admin_assigned
2024-01-29T11:25:08.175Z
63ca3ddc04c979828310bfcb
/avatars/615e0d8622950b4408b40d550f02a894.svg
Ying Shan
user
yshan2u
TIP-Editor: An Accurate 3D Editor Following Both Text-Prompts And
  Image-Prompts
Text-driven 3D scene editing has gained significant attention owing to its
convenience and user-friendliness. However, existing methods still lack
accurate control of the specified appearance and location of the editing result
due to the inherent limitations of the text description. To this end, we
propose a 3D scene editing framework, TIPEditor, that accepts both text and
image prompts and a 3D bounding box to specify the editing region. With the
image prompt, users can conveniently specify the detailed appearance/style of
the target content in complement to the text description, enabling accurate
control of the appearance. Specifically, TIP-Editor employs a stepwise 2D
personalization strategy to better learn the representation of the existing
scene and the reference image, in which a localization loss is proposed to
encourage correct object placement as specified by the bounding box.
Additionally, TIPEditor utilizes explicit and flexible 3D Gaussian splatting as
the 3D representation to facilitate local editing while keeping the background
unchanged. Extensive experiments have demonstrated that TIP-Editor conducts
accurate editing following the text and image prompts in the specified bounding
box region, consistently outperforming the baselines in editing quality, and
the alignment to the prompts, qualitatively and quantitatively.
TIPEditor, a text- and image-driven 3D scene editing framework, uses 2D personalization and 3D Gaussian splatting to achieve accurate and aligned edits within specified bounding boxes.
3D scene editing text-driven image prompts 3D bounding box 2D personalization localization loss 3D Gaussian splatting