title: DORSal: Diffusion for Object-centric Representations of Scenes
    et al.
summary: Recent progress in 3D scene understanding enables scalable learning of
  representations across large datasets of diverse scenes. As a consequence,
  generalization to unseen scenes and objects, rendering novel views from just a
  single or a handful of input images, and controllable scene generation that
  supports editing, is now possible. However, training jointly on a large number
  of scenes typically compromises rendering quality when compared to single-scene
  optimized models such as NeRFs. In this paper, we leverage recent progress in
  diffusion models to equip 3D scene representation learning models with the
  ability to render high-fidelity novel views, while retaining benefits such as
  object-level scene editing to a large degree. In particular, we propose DORSal,
  which adapts a video diffusion architecture for 3D scene generation conditioned
  on object-centric slot-based representations of scenes. On both complex
  synthetic multi-object scenes and on the real-world large-scale Street View
  dataset, we show that DORSal enables scalable neural rendering of 3D scenes
  with object-level editing and improves upon existing approaches.
paper_authors._id: 648bf1bc335645aa03f03965
paper_authors.name: Allan Jabri
paper_authors._id: 648bf1bc335645aa03f03966
paper_authors.name: Sjoerd van Steenkiste
paper_authors._id: 648bf1bc335645aa03f03967
paper_authors.name: Emiel Hoogeboom
paper_authors._id: 648bf1bc335645aa03f03968
paper_authors.name: Mehdi S. M. Sajjadi
paper_authors._id: 648bf1bc335645aa03f03969
paper_authors.name: Thomas Kipf
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-06-16T10:52:05.911Z
paper_authors._id: 6413572b6cd62eb3ba2024e9
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6413572b6cd62eb3ba2024e9/UCtMdeV8_N6w-SBnTDXBy.jpeg
paper_authors.fullname: Thomas Kipf
paper_authors.type: user
paper_authors.user: tkipf
paper_title: DORSal: Diffusion for Object-centric Representations of Scenes
    et al.
paper_summary: Recent progress in 3D scene understanding enables scalable learning of
  representations across large datasets of diverse scenes. As a consequence,
  generalization to unseen scenes and objects, rendering novel views from just a
  single or a handful of input images, and controllable scene generation that
  supports editing, is now possible. However, training jointly on a large number
  of scenes typically compromises rendering quality when compared to single-scene
  optimized models such as NeRFs. In this paper, we leverage recent progress in
  diffusion models to equip 3D scene representation learning models with the
  ability to render high-fidelity novel views, while retaining benefits such as
  object-level scene editing to a large degree. In particular, we propose DORSal,
  which adapts a video diffusion architecture for 3D scene generation conditioned
  on object-centric slot-based representations of scenes. On both complex
  synthetic multi-object scenes and on the real-world large-scale Street View
  dataset, we show that DORSal enables scalable neural rendering of 3D scenes
  with object-level editing and improves upon existing approaches.
paper_ai_summary: A video diffusion architecture adapted for 3D scene generation improves rendering quality and scalability while maintaining object-level editing capabilities.
paper_ai_keywords: diffusion models DORSal video diffusion architecture 3D scene generation object-centric slot-based representations neural rendering