title: Mosaic-SDF for 3D Generative Models
summary: Current diffusion or flow-based generative models for 3D shapes divide to
  two: distilling pre-trained 2D image diffusion models, and training directly on
  3D shapes. When training a diffusion or flow models on 3D shapes a crucial
  design choice is the shape representation. An effective shape representation
  needs to adhere three design principles: it should allow an efficient
  conversion of large 3D datasets to the representation form; it should provide a
  good tradeoff of approximation power versus number of parameters; and it should
  have a simple tensorial form that is compatible with existing powerful neural
  architectures. While standard 3D shape representations such as volumetric grids
  and point clouds do not adhere to all these principles simultaneously, we
  advocate in this paper a new representation that does. We introduce Mosaic-SDF
  (M-SDF): a simple 3D shape representation that approximates the Signed Distance
  Function (SDF) of a given shape by using a set of local grids spread near the
  shape's boundary. The M-SDF representation is fast to compute for each shape
  individually making it readily parallelizable; it is parameter efficient as it
  only covers the space around the shape's boundary; and it has a simple matrix
  form, compatible with Transformer-based architectures. We demonstrate the
  efficacy of the M-SDF representation by using it to train a 3D generative flow
  model including class-conditioned generation with the 3D Warehouse dataset, and
  text-to-3D generation using a dataset of about 600k caption-shape pairs.
paper_authors._id: 657bd80564dab2f4c9fcf770
paper_authors.name: Lior Yariv
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-15T11:10:16.637Z
paper_authors._id: 63f351070be81bdc5d90fd61
paper_authors.avatarUrl: /avatars/752d8bc512cdad704bc814876514bc87.svg
paper_authors.fullname: Lior Yariv
paper_authors.type: user
paper_authors.user: lioryariv
paper_authors._id: 657bd80564dab2f4c9fcf771
paper_authors.name: Omri Puny
paper_authors._id: 657bd80564dab2f4c9fcf772
paper_authors.name: Natalia Neverova
paper_authors._id: 657bd80564dab2f4c9fcf773
paper_authors.name: Oran Gafni
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-15T11:09:42.274Z
paper_authors._id: 6345a97d547c70e4b7ce9169
paper_authors.avatarUrl: /avatars/5d233e65eadfa7d9ef0390a584df5451.svg
paper_authors.fullname: Oran Gafni
paper_authors.type: user
paper_authors.user: Oran22
paper_authors._id: 657bd80564dab2f4c9fcf774
paper_authors.name: Yaron Lipman
paper_title: Mosaic-SDF for 3D Generative Models
paper_summary: Current diffusion or flow-based generative models for 3D shapes divide to
  two: distilling pre-trained 2D image diffusion models, and training directly on
  3D shapes. When training a diffusion or flow models on 3D shapes a crucial
  design choice is the shape representation. An effective shape representation
  needs to adhere three design principles: it should allow an efficient
  conversion of large 3D datasets to the representation form; it should provide a
  good tradeoff of approximation power versus number of parameters; and it should
  have a simple tensorial form that is compatible with existing powerful neural
  architectures. While standard 3D shape representations such as volumetric grids
  and point clouds do not adhere to all these principles simultaneously, we
  advocate in this paper a new representation that does. We introduce Mosaic-SDF
  (M-SDF): a simple 3D shape representation that approximates the Signed Distance
  Function (SDF) of a given shape by using a set of local grids spread near the
  shape's boundary. The M-SDF representation is fast to compute for each shape
  individually making it readily parallelizable; it is parameter efficient as it
  only covers the space around the shape's boundary; and it has a simple matrix
  form, compatible with Transformer-based architectures. We demonstrate the
  efficacy of the M-SDF representation by using it to train a 3D generative flow
  model including class-conditioned generation with the 3D Warehouse dataset, and
  text-to-3D generation using a dataset of about 600k caption-shape pairs.
paper_ai_summary: A new Mosaic-SDF representation for 3D shapes enables efficient, parameter-efficient, and parallelizable generation of 3D shapes using generative flow models.
paper_ai_keywords: diffusion models flow-based generative models 3D shapes shape representation volumetric grids point clouds Signed Distance Function (SDF) Mosaic-SDF Transformer-based architectures class-conditioned generation text-to-3D generation