Find Any Part in 3D
We study open-world part segmentation in 3D: segmenting any part in any
object based on any text query. Prior methods are limited in object categories
and part vocabularies. Recent advances in AI have demonstrated effective
open-world recognition capabilities in 2D. Inspired by this progress, we
propose an open-world, direct-prediction model for 3D part segmentation that
can be applied zero-shot to any object. Our approach, called Find3D, trains a
general-category point embedding model on large-scale 3D assets from the
internet without any human annotation. It combines a data engine, powered by
foundation models for annotating data, with a contrastive training method. We
achieve strong performance and generalization across multiple datasets, with up
to a 3x improvement in mIoU over the next best method. Our model is 6x to over
300x faster than existing baselines. To encourage research in general-category
open-world 3D part segmentation, we also release a benchmark for general
objects and parts. Project website: https://ziqi-ma.github.io/find3dsite/
673f0a21c46b16fa202d60c8
Ziqi Ma
admin_assigned
2024-11-26T22:53:23.057Z
672fba6df51af868287e029b
https://cdn-avatars.huggingface.co/v1/production/uploads/672fba6df51af868287e029b/cdwIFU3U0uk_s7AOn90fF.jpeg
Ziqi
user
ziqima
673f0a21c46b16fa202d60c9
Yisong Yue
admin_assigned
2024-11-26T22:52:57.125Z
6350307b14fb199c76589589
/avatars/41ed1cda4fc5a37e98c5fdb3fdbfe09c.svg
Yisong
user
yisongyue
673f0a21c46b16fa202d60ca
Georgia Gkioxari
admin_assigned
2024-11-26T22:52:51.130Z
634ededbd049354d7ee4b557
https://cdn-avatars.huggingface.co/v1/production/uploads/634ededbd049354d7ee4b557/XJS3qqpExVDFCKjT0NMi8.jpeg
Georgia Gkioxari
user
gkioxari
Find Any Part in 3D
We study open-world part segmentation in 3D: segmenting any part in any
object based on any text query. Prior methods are limited in object categories
and part vocabularies. Recent advances in AI have demonstrated effective
open-world recognition capabilities in 2D. Inspired by this progress, we
propose an open-world, direct-prediction model for 3D part segmentation that
can be applied zero-shot to any object. Our approach, called Find3D, trains a
general-category point embedding model on large-scale 3D assets from the
internet without any human annotation. It combines a data engine, powered by
foundation models for annotating data, with a contrastive training method. We
achieve strong performance and generalization across multiple datasets, with up
to a 3x improvement in mIoU over the next best method. Our model is 6x to over
300x faster than existing baselines. To encourage research in general-category
open-world 3D part segmentation, we also release a benchmark for general
objects and parts. Project website: https://ziqi-ma.github.io/find3dsite/
A model named Find3D achieves strong performance and generalization in open-world 3D part segmentation using a combination of foundation models and contrastive training without human annotation, outperforming existing methods in speed and accuracy.
open-world part segmentation point embedding data engine foundation models contrastive training mIoU general-category benchmark