DreamMesh: Jointly Manipulating and Texturing Triangle Meshes for Text-to-3D Generation
<|reference_start|>DreamMesh: Jointly Manipulating and Texturing Triangle Meshes for Text-to-3D Generation: Learning radiance fields (NeRF) with powerful 2D diffusion models has garnered popularity for text-to-3D generation. Nevertheless, the implicit 3D representations of NeRF lack explicit modeling of meshes and textures over surfaces, and such surface-undefined way may suffer from the issues, e.g., noisy surfaces with ambiguous texture details or cross-view inconsistency. To alleviate this, we present DreamMesh, a novel text-to-3D architecture that pivots on well-defined surfaces (triangle meshes) to generate high-fidelity explicit 3D model. Technically, DreamMesh capitalizes on a distinctive coarse-to-fine scheme. In the coarse stage, the mesh is first deformed by text-guided Jacobians and then DreamMesh textures the mesh with an interlaced use of 2D diffusion models in a tuning free manner from multiple viewpoints. In the fine stage, DreamMesh jointly manipulates the mesh and refines the texture map, leading to high-quality triangle meshes with high-fidelity textured materials. Extensive experiments demonstrate that DreamMesh significantly outperforms state-of-the-art text-to-3D methods in faithfully generating 3D content with richer textual details and enhanced geometry. Our project page is available at https://dreammesh.github.io.<|reference_end|>
@article{yang2024dreammesh:,
    title={DreamMesh: Jointly Manipulating and Texturing Triangle Meshes for
  Text-to-3D Generation},
    author={Haibo Yang and Yang Chen and Yingwei Pan and Ting Yao and Zhineng Chen
  and Zuxuan Wu and Yu-Gang Jiang and Tao Mei},
    journal={arXiv preprint arXiv:2409.07454},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.07454},
    primaryClass={cs.CV cs.MM}
}