title: Compress3D: a Compressed Latent Space for 3D Generation from a Single
    Image
summary: 3D generation has witnessed significant advancements, yet efficiently
  producing high-quality 3D assets from a single image remains challenging. In
  this paper, we present a triplane autoencoder, which encodes 3D models into a
  compact triplane latent space to effectively compress both the 3D geometry and
  texture information. Within the autoencoder framework, we introduce a 3D-aware
  cross-attention mechanism, which utilizes low-resolution latent representations
  to query features from a high-resolution 3D feature volume, thereby enhancing
  the representation capacity of the latent space. Subsequently, we train a
  diffusion model on this refined latent space. In contrast to solely relying on
  image embedding for 3D generation, our proposed method advocates for the
  simultaneous utilization of both image embedding and shape embedding as
  conditions. Specifically, the shape embedding is estimated via a diffusion
  prior model conditioned on the image embedding. Through comprehensive
  experiments, we demonstrate that our method outperforms state-of-the-art
  algorithms, achieving superior performance while requiring less training data
  and time. Our approach enables the generation of high-quality 3D assets in
  merely 7 seconds on a single A100 GPU.
paper_authors._id: 65fba5f7dfd0ff86aa77dd44
paper_authors.name: Bowen Zhang
paper_authors._id: 65fba5f7dfd0ff86aa77dd45
paper_authors.name: Tianyu Yang
paper_authors._id: 65fba5f7dfd0ff86aa77dd46
paper_authors.name: Yu Li
paper_authors._id: 65fba5f7dfd0ff86aa77dd47
paper_authors.name: Lei Zhang
paper_authors._id: 65fba5f7dfd0ff86aa77dd48
paper_authors.name: Xi Zhao
paper_title: Compress3D: a Compressed Latent Space for 3D Generation from a Single
    Image
paper_summary: 3D generation has witnessed significant advancements, yet efficiently
  producing high-quality 3D assets from a single image remains challenging. In
  this paper, we present a triplane autoencoder, which encodes 3D models into a
  compact triplane latent space to effectively compress both the 3D geometry and
  texture information. Within the autoencoder framework, we introduce a 3D-aware
  cross-attention mechanism, which utilizes low-resolution latent representations
  to query features from a high-resolution 3D feature volume, thereby enhancing
  the representation capacity of the latent space. Subsequently, we train a
  diffusion model on this refined latent space. In contrast to solely relying on
  image embedding for 3D generation, our proposed method advocates for the
  simultaneous utilization of both image embedding and shape embedding as
  conditions. Specifically, the shape embedding is estimated via a diffusion
  prior model conditioned on the image embedding. Through comprehensive
  experiments, we demonstrate that our method outperforms state-of-the-art
  algorithms, achieving superior performance while requiring less training data
  and time. Our approach enables the generation of high-quality 3D assets in
  merely 7 seconds on a single A100 GPU.
paper_ai_summary: A triplane autoencoder with a 3D-aware cross-attention mechanism and diffusion model outperforms existing methods in generating high-quality 3D assets from a single image, using both image and shape embeddings.
paper_ai_keywords: triplane autoencoder triplane latent space 3D-aware cross-attention mechanism high-resolution 3D feature volume diffusion model image embedding shape embedding diffusion prior model A100 GPU