title: MARVEL-40M+: Multi-Level Visual Elaboration for High-Fidelity Text-to-3D
    Content Creation
summary: Generating high-fidelity 3D content from text prompts remains a significant
  challenge in computer vision due to the limited size, diversity, and annotation
  depth of the existing datasets. To address this, we introduce MARVEL-40M+, an
  extensive dataset with 40 million text annotations for over 8.9 million 3D
  assets aggregated from seven major 3D datasets. Our contribution is a novel
  multi-stage annotation pipeline that integrates open-source pretrained
  multi-view VLMs and LLMs to automatically produce multi-level descriptions,
  ranging from detailed (150-200 words) to concise semantic tags (10-20 words).
  This structure supports both fine-grained 3D reconstruction and rapid
  prototyping. Furthermore, we incorporate human metadata from source datasets
  into our annotation pipeline to add domain-specific information in our
  annotation and reduce VLM hallucinations. Additionally, we develop MARVEL-FX3D,
  a two-stage text-to-3D pipeline. We fine-tune Stable Diffusion with our
  annotations and use a pretrained image-to-3D network to generate 3D textured
  meshes within 15s. Extensive evaluations show that MARVEL-40M+ significantly
  outperforms existing datasets in annotation quality and linguistic diversity,
  achieving win rates of 72.41% by GPT-4 and 73.40% by human evaluators.
paper_authors._id: 6747c9cfae9bfb99e7596c3f
paper_authors.name: Sankalp Sinha
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-29T10:08:12.351Z
paper_authors._id: 65e0c058bd0c59bbf2708e32
paper_authors.avatarUrl: /avatars/a23844839af08f23b08f537c1c39dc03.svg
paper_authors.fullname: Sankalp Sinha
paper_authors.type: user
paper_authors.user: sankalpsinha77
paper_authors._id: 6747c9cfae9bfb99e7596c40
paper_authors.name: Mohammad Sadil Khan
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-28T10:35:09.219Z
paper_authors._id: 65e07fb40d364b871d406648
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/65e07fb40d364b871d406648/ERg4Te5DB8e__ZJuj9E1m.png
paper_authors.fullname: Mohammad Sadil Khan
paper_authors.type: user
paper_authors.user: SadilKhan
paper_authors._id: 6747c9cfae9bfb99e7596c41
paper_authors.name: Muhammad Usama
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-28T12:29:26.415Z
paper_authors._id: 6623d8266046dd4b8575906f
paper_authors.avatarUrl: /avatars/0ca98a2b24f598ce02bf23ddbc6a7ecf.svg
paper_authors.fullname: Muhammad Usama
paper_authors.type: user
paper_authors.user: MUsama100
paper_authors._id: 6747c9cfae9bfb99e7596c42
paper_authors.name: Shino Sam
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-28T10:41:32.383Z
paper_authors._id: 66576f6e644a2dd7f2f93d68
paper_authors.avatarUrl: /avatars/7da4c7bc2299b4216ffc0aa65cef0a26.svg
paper_authors.fullname: Shino Sam
paper_authors.type: user
paper_authors.user: alootikki
paper_authors._id: 6747c9cfae9bfb99e7596c43
paper_authors.name: Didier Stricker
paper_authors._id: 6747c9cfae9bfb99e7596c44
paper_authors.name: Sk Aziz Ali
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-28T10:41:17.052Z
paper_authors._id: 65a11c41712e1321d051617b
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/pWfZoI9PnErCKOf5XFiN1.jpeg
paper_authors.fullname: Sk Aziz Ali
paper_authors.type: user
paper_authors.user: saali14
paper_authors._id: 6747c9cfae9bfb99e7596c45
paper_authors.name: Muhammad Zeshan Afzal
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-28T10:40:52.954Z
paper_authors._id: 663b5392e6257fa86a26e8ef
paper_authors.avatarUrl: /avatars/5a94f3ea7b0bd60264c35643de5423a0.svg
paper_authors.fullname: Muhammad Zeshan Afzal
paper_authors.type: user
paper_authors.user: mzafzal
paper_title: MARVEL-40M+: Multi-Level Visual Elaboration for High-Fidelity Text-to-3D
    Content Creation
paper_summary: Generating high-fidelity 3D content from text prompts remains a significant
  challenge in computer vision due to the limited size, diversity, and annotation
  depth of the existing datasets. To address this, we introduce MARVEL-40M+, an
  extensive dataset with 40 million text annotations for over 8.9 million 3D
  assets aggregated from seven major 3D datasets. Our contribution is a novel
  multi-stage annotation pipeline that integrates open-source pretrained
  multi-view VLMs and LLMs to automatically produce multi-level descriptions,
  ranging from detailed (150-200 words) to concise semantic tags (10-20 words).
  This structure supports both fine-grained 3D reconstruction and rapid
  prototyping. Furthermore, we incorporate human metadata from source datasets
  into our annotation pipeline to add domain-specific information in our
  annotation and reduce VLM hallucinations. Additionally, we develop MARVEL-FX3D,
  a two-stage text-to-3D pipeline. We fine-tune Stable Diffusion with our
  annotations and use a pretrained image-to-3D network to generate 3D textured
  meshes within 15s. Extensive evaluations show that MARVEL-40M+ significantly
  outperforms existing datasets in annotation quality and linguistic diversity,
  achieving win rates of 72.41% by GPT-4 and 73.40% by human evaluators.
paper_ai_summary: A novel dataset MARVEL-40M+ with extensive text annotations and a two-stage pipeline for high-fidelity 3D content generation from text prompts outperforms existing datasets in annotation quality and diversity.
paper_ai_keywords: multi-stage annotation pipeline multi-view VLMs LLMs MARVEL-FX3D Stable Diffusion image-to-3D network 3D textured meshes win rates linguistic diversity