Feed-Forward SceneDINO for Unsupervised Semantic Scene Completion
Semantic scene completion (SSC) aims to infer both the 3D geometry and
semantics of a scene from single images. In contrast to prior work on SSC that
heavily relies on expensive ground-truth annotations, we approach SSC in an
unsupervised setting. Our novel method, SceneDINO, adapts techniques from
self-supervised representation learning and 2D unsupervised scene understanding
to SSC. Our training exclusively utilizes multi-view consistency
self-supervision without any form of semantic or geometric ground truth. Given
a single input image, SceneDINO infers the 3D geometry and expressive 3D DINO
features in a feed-forward manner. Through a novel 3D feature distillation
approach, we obtain unsupervised 3D semantics. In both 3D and 2D unsupervised
scene understanding, SceneDINO reaches state-of-the-art segmentation accuracy.
Linear probing our 3D features matches the segmentation accuracy of a current
supervised SSC approach. Additionally, we showcase the domain generalization
and multi-view consistency of SceneDINO, taking the first steps towards a
strong foundation for single image 3D scene understanding.
686dfd05cb5725779c60b4c8
Aleksandar Jevtić
claimed_verified
2025-07-09T08:49:39.517Z
685e6ef1882f117bc2b7d981
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/MU2Bgys-5qADLivaU__BX.jpeg
Aleksandar Jevtić
user
jev-aleks
686dfd05cb5725779c60b4c9
Christoph Reich
claimed_verified
2025-07-09T14:40:58.573Z
67f001a5f824e54a66563455
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/AG6K_dOseLcc84tHP7V8D.png
Christoph Reich
user
ChristophReich1996
686dfd05cb5725779c60b4ca
Felix Wimbauer
686dfd05cb5725779c60b4cb
Oliver Hahn
claimed_verified
2025-07-09T14:40:56.536Z
636020de88e41d249eccf68c
https://cdn-avatars.huggingface.co/v1/production/uploads/636020de88e41d249eccf68c/pMduOM5UNZo4JW2Ckk2jK.jpeg
Oliver Hahn
user
olvrhhn
686dfd05cb5725779c60b4cc
Christian Rupprecht
686dfd05cb5725779c60b4cd
Stefan Roth
686dfd05cb5725779c60b4ce
Daniel Cremers
Feed-Forward SceneDINO for Unsupervised Semantic Scene Completion
Semantic scene completion (SSC) aims to infer both the 3D geometry and
semantics of a scene from single images. In contrast to prior work on SSC that
heavily relies on expensive ground-truth annotations, we approach SSC in an
unsupervised setting. Our novel method, SceneDINO, adapts techniques from
self-supervised representation learning and 2D unsupervised scene understanding
to SSC. Our training exclusively utilizes multi-view consistency
self-supervision without any form of semantic or geometric ground truth. Given
a single input image, SceneDINO infers the 3D geometry and expressive 3D DINO
features in a feed-forward manner. Through a novel 3D feature distillation
approach, we obtain unsupervised 3D semantics. In both 3D and 2D unsupervised
scene understanding, SceneDINO reaches state-of-the-art segmentation accuracy.
Linear probing our 3D features matches the segmentation accuracy of a current
supervised SSC approach. Additionally, we showcase the domain generalization
and multi-view consistency of SceneDINO, taking the first steps towards a
strong foundation for single image 3D scene understanding.
SceneDINO achieves state-of-the-art segmentation accuracy in unsupervised 3D scene completion by leveraging self-supervised learning and 3D feature distillation.
self-supervised representation learning 2D unsupervised scene understanding multi-view consistency feed-forward manner 3D feature distillation domain generalization