title: Scenethesis: A Language and Vision Agentic Framework for 3D Scene
    Generation
summary: Synthesizing interactive 3D scenes from text is essential for gaming, virtual
  reality, and embodied AI. However, existing methods face several challenges.
  Learning-based approaches depend on small-scale indoor datasets, limiting the
  scene diversity and layout complexity. While large language models (LLMs) can
  leverage diverse text-domain knowledge, they struggle with spatial realism,
  often producing unnatural object placements that fail to respect common sense.
  Our key insight is that vision perception can bridge this gap by providing
  realistic spatial guidance that LLMs lack. To this end, we introduce
  Scenethesis, a training-free agentic framework that integrates LLM-based scene
  planning with vision-guided layout refinement. Given a text prompt, Scenethesis
  first employs an LLM to draft a coarse layout. A vision module then refines it
  by generating an image guidance and extracting scene structure to capture
  inter-object relations. Next, an optimization module iteratively enforces
  accurate pose alignment and physical plausibility, preventing artifacts like
  object penetration and instability. Finally, a judge module verifies spatial
  coherence. Comprehensive experiments show that Scenethesis generates diverse,
  realistic, and physically plausible 3D interactive scenes, making it valuable
  for virtual content creation, simulation environments, and embodied AI
  research.
paper_authors._id: 6819a5106d73cfa9574a900a
paper_authors.name: Lu Ling
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-06T08:32:27.485Z
paper_authors._id: 6347935d79e42766e212fa53
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6347935d79e42766e212fa53/9BBADaR0yV1wjC91pGYmZ.png
paper_authors.fullname: Lu Ling
paper_authors.type: user
paper_authors.user: LuLing
paper_authors._id: 6819a5106d73cfa9574a900b
paper_authors.name: Chen-Hsuan Lin
paper_authors._id: 6819a5106d73cfa9574a900c
paper_authors.name: Tsung-Yi Lin
paper_authors._id: 6819a5106d73cfa9574a900d
paper_authors.name: Yifan Ding
paper_authors._id: 6819a5106d73cfa9574a900e
paper_authors.name: Yu Zeng
paper_authors._id: 6819a5106d73cfa9574a900f
paper_authors.name: Yichen Sheng
paper_authors._id: 6819a5106d73cfa9574a9010
paper_authors.name: Yunhao Ge
paper_authors._id: 6819a5106d73cfa9574a9011
paper_authors.name: Ming-Yu Liu
paper_authors._id: 6819a5106d73cfa9574a9012
paper_authors.name: Aniket Bera
paper_authors._id: 6819a5106d73cfa9574a9013
paper_authors.name: Zhaoshuo Li
paper_title: Scenethesis: A Language and Vision Agentic Framework for 3D Scene
    Generation
paper_summary: Synthesizing interactive 3D scenes from text is essential for gaming, virtual
  reality, and embodied AI. However, existing methods face several challenges.
  Learning-based approaches depend on small-scale indoor datasets, limiting the
  scene diversity and layout complexity. While large language models (LLMs) can
  leverage diverse text-domain knowledge, they struggle with spatial realism,
  often producing unnatural object placements that fail to respect common sense.
  Our key insight is that vision perception can bridge this gap by providing
  realistic spatial guidance that LLMs lack. To this end, we introduce
  Scenethesis, a training-free agentic framework that integrates LLM-based scene
  planning with vision-guided layout refinement. Given a text prompt, Scenethesis
  first employs an LLM to draft a coarse layout. A vision module then refines it
  by generating an image guidance and extracting scene structure to capture
  inter-object relations. Next, an optimization module iteratively enforces
  accurate pose alignment and physical plausibility, preventing artifacts like
  object penetration and instability. Finally, a judge module verifies spatial
  coherence. Comprehensive experiments show that Scenethesis generates diverse,
  realistic, and physically plausible 3D interactive scenes, making it valuable
  for virtual content creation, simulation environments, and embodied AI
  research.
paper_ai_summary: Scenethesis combines language models with vision-guided refinement to create diverse, realistic, and physically plausible 3D scenes from text.
paper_ai_keywords: LLMs vision perception Scenethesis agentic framework scene planning vision module image guidance scene structure inter-object relations optimization module pose alignment physical plausibility judge module spatial coherence 3D scenes virtual content creation simulation environments embodied AI