MARVEL-40M+: Multi-Level Visual Elaboration for High-Fidelity Text-to-3D
  Content Creation
Generating high-fidelity 3D content from text prompts remains a significant
challenge in computer vision due to the limited size, diversity, and annotation
depth of the existing datasets. To address this, we introduce MARVEL-40M+, an
extensive dataset with 40 million text annotations for over 8.9 million 3D
assets aggregated from seven major 3D datasets. Our contribution is a novel
multi-stage annotation pipeline that integrates open-source pretrained
multi-view VLMs and LLMs to automatically produce multi-level descriptions,
ranging from detailed (150-200 words) to concise semantic tags (10-20 words).
This structure supports both fine-grained 3D reconstruction and rapid
prototyping. Furthermore, we incorporate human metadata from source datasets
into our annotation pipeline to add domain-specific information in our
annotation and reduce VLM hallucinations. Additionally, we develop MARVEL-FX3D,
a two-stage text-to-3D pipeline. We fine-tune Stable Diffusion with our
annotations and use a pretrained image-to-3D network to generate 3D textured
meshes within 15s. Extensive evaluations show that MARVEL-40M+ significantly
outperforms existing datasets in annotation quality and linguistic diversity,
achieving win rates of 72.41% by GPT-4 and 73.40% by human evaluators.
6747c9cfae9bfb99e7596c3f
Sankalp Sinha
claimed_verified
2024-11-29T10:08:12.351Z
65e0c058bd0c59bbf2708e32
/avatars/a23844839af08f23b08f537c1c39dc03.svg
Sankalp Sinha
user
sankalpsinha77
6747c9cfae9bfb99e7596c40
Mohammad Sadil Khan
claimed_verified
2024-11-28T10:35:09.219Z
65e07fb40d364b871d406648
https://cdn-avatars.huggingface.co/v1/production/uploads/65e07fb40d364b871d406648/ERg4Te5DB8e__ZJuj9E1m.png
Mohammad Sadil Khan
user
SadilKhan
6747c9cfae9bfb99e7596c41
Muhammad Usama
claimed_verified
2024-11-28T12:29:26.415Z
6623d8266046dd4b8575906f
/avatars/0ca98a2b24f598ce02bf23ddbc6a7ecf.svg
Muhammad Usama
user
MUsama100
6747c9cfae9bfb99e7596c42
Shino Sam
admin_assigned
2024-11-28T10:41:32.383Z
66576f6e644a2dd7f2f93d68
/avatars/7da4c7bc2299b4216ffc0aa65cef0a26.svg
Shino Sam
user
alootikki
6747c9cfae9bfb99e7596c43
Didier Stricker
6747c9cfae9bfb99e7596c44
Sk Aziz Ali
admin_assigned
2024-11-28T10:41:17.052Z
65a11c41712e1321d051617b
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/pWfZoI9PnErCKOf5XFiN1.jpeg
Sk Aziz Ali
user
saali14
6747c9cfae9bfb99e7596c45
Muhammad Zeshan Afzal
admin_assigned
2024-11-28T10:40:52.954Z
663b5392e6257fa86a26e8ef
/avatars/5a94f3ea7b0bd60264c35643de5423a0.svg
Muhammad Zeshan Afzal
user
mzafzal
MARVEL-40M+: Multi-Level Visual Elaboration for High-Fidelity Text-to-3D
  Content Creation
Generating high-fidelity 3D content from text prompts remains a significant
challenge in computer vision due to the limited size, diversity, and annotation
depth of the existing datasets. To address this, we introduce MARVEL-40M+, an
extensive dataset with 40 million text annotations for over 8.9 million 3D
assets aggregated from seven major 3D datasets. Our contribution is a novel
multi-stage annotation pipeline that integrates open-source pretrained
multi-view VLMs and LLMs to automatically produce multi-level descriptions,
ranging from detailed (150-200 words) to concise semantic tags (10-20 words).
This structure supports both fine-grained 3D reconstruction and rapid
prototyping. Furthermore, we incorporate human metadata from source datasets
into our annotation pipeline to add domain-specific information in our
annotation and reduce VLM hallucinations. Additionally, we develop MARVEL-FX3D,
a two-stage text-to-3D pipeline. We fine-tune Stable Diffusion with our
annotations and use a pretrained image-to-3D network to generate 3D textured
meshes within 15s. Extensive evaluations show that MARVEL-40M+ significantly
outperforms existing datasets in annotation quality and linguistic diversity,
achieving win rates of 72.41% by GPT-4 and 73.40% by human evaluators.
A novel dataset MARVEL-40M+ with extensive text annotations and a two-stage pipeline for high-fidelity 3D content generation from text prompts outperforms existing datasets in annotation quality and diversity.
multi-stage annotation pipeline multi-view VLMs LLMs MARVEL-FX3D Stable Diffusion image-to-3D network 3D textured meshes win rates linguistic diversity