InstructScene: Instruction-Driven 3D Indoor Scene Synthesis with Semantic Graph Prior
<|reference_start|>InstructScene: Instruction-Driven 3D Indoor Scene Synthesis with Semantic Graph Prior: Comprehending natural language instructions is a charming property for 3D indoor scene synthesis systems. Existing methods directly model object joint distributions and express object relations implicitly within a scene, thereby hindering the controllability of generation. We introduce InstructScene, a novel generative framework that integrates a semantic graph prior and a layout decoder to improve controllability and fidelity for 3D scene synthesis. The proposed semantic graph prior jointly learns scene appearances and layout distributions, exhibiting versatility across various downstream tasks in a zero-shot manner. To facilitate the benchmarking for text-driven 3D scene synthesis, we curate a high-quality dataset of scene-instruction pairs with large language and multimodal models. Extensive experimental results reveal that the proposed method surpasses existing state-of-the-art approaches by a large margin. Thorough ablation studies confirm the efficacy of crucial design components. Project page: https://chenguolin.github.io/projects/InstructScene.<|reference_end|>
@article{lin2024instructscene:,
    title={InstructScene: Instruction-Driven 3D Indoor Scene Synthesis with
  Semantic Graph Prior},
    author={Chenguo Lin, Yadong Mu},
    journal={arXiv preprint arXiv:2402.04717},
    year={2024},
    archivePrefix={arXiv},
    eprint={2402.04717},
    primaryClass={cs.CV}
}