Click-Gaussian: Interactive Segmentation to Any 3D Gaussians
Interactive segmentation of 3D Gaussians opens a great opportunity for
real-time manipulation of 3D scenes thanks to the real-time rendering
capability of 3D Gaussian Splatting. However, the current methods suffer from
time-consuming post-processing to deal with noisy segmentation output. Also,
they struggle to provide detailed segmentation, which is important for
fine-grained manipulation of 3D scenes. In this study, we propose
Click-Gaussian, which learns distinguishable feature fields of two-level
granularity, facilitating segmentation without time-consuming post-processing.
We delve into challenges stemming from inconsistently learned feature fields
resulting from 2D segmentation obtained independently from a 3D scene. 3D
segmentation accuracy deteriorates when 2D segmentation results across the
views, primary cues for 3D segmentation, are in conflict. To overcome these
issues, we propose Global Feature-guided Learning (GFL). GFL constructs the
clusters of global feature candidates from noisy 2D segments across the views,
which smooths out noises when training the features of 3D Gaussians. Our method
runs in 10 ms per click, 15 to 130 times as fast as the previous methods, while
also significantly improving segmentation accuracy. Our project page is
available at https://seokhunchoi.github.io/Click-Gaussian
66973b46c1d9adc0c7d6b0d1
Seokhun Choi
66973b46c1d9adc0c7d6b0d2
Hyeonseop Song
66973b46c1d9adc0c7d6b0d3
Jaechul Kim
66973b46c1d9adc0c7d6b0d4
Taehyeong Kim
66973b46c1d9adc0c7d6b0d5
Hoseok Do
Click-Gaussian: Interactive Segmentation to Any 3D Gaussians
Interactive segmentation of 3D Gaussians opens a great opportunity for
real-time manipulation of 3D scenes thanks to the real-time rendering
capability of 3D Gaussian Splatting. However, the current methods suffer from
time-consuming post-processing to deal with noisy segmentation output. Also,
they struggle to provide detailed segmentation, which is important for
fine-grained manipulation of 3D scenes. In this study, we propose
Click-Gaussian, which learns distinguishable feature fields of two-level
granularity, facilitating segmentation without time-consuming post-processing.
We delve into challenges stemming from inconsistently learned feature fields
resulting from 2D segmentation obtained independently from a 3D scene. 3D
segmentation accuracy deteriorates when 2D segmentation results across the
views, primary cues for 3D segmentation, are in conflict. To overcome these
issues, we propose Global Feature-guided Learning (GFL). GFL constructs the
clusters of global feature candidates from noisy 2D segments across the views,
which smooths out noises when training the features of 3D Gaussians. Our method
runs in 10 ms per click, 15 to 130 times as fast as the previous methods, while
also significantly improving segmentation accuracy. Our project page is
available at https://seokhunchoi.github.io/Click-Gaussian
Click-Gaussian improves real-time 3D segmentation accuracy by learning distinguishable feature fields and using global feature-guided learning to address inconsistencies from 2D views.
3D Gaussian Splatting Click-Gaussian feature fields global feature-guided learning GFL 3D segmentation 2D segmentation feature clustering