title: 3D-GPT: Procedural 3D Modeling with Large Language Models
summary: In the pursuit of efficient automated content creation, procedural
  generation, leveraging modifiable parameters and rule-based systems, emerges as
  a promising approach. Nonetheless, it could be a demanding endeavor, given its
  intricate nature necessitating a deep understanding of rules, algorithms, and
  parameters. To reduce workload, we introduce 3D-GPT, a framework utilizing
  large language models~(LLMs) for instruction-driven 3D modeling. 3D-GPT
  positions LLMs as proficient problem solvers, dissecting the procedural 3D
  modeling tasks into accessible segments and appointing the apt agent for each
  task. 3D-GPT integrates three core agents: the task dispatch agent, the
  conceptualization agent, and the modeling agent. They collaboratively achieve
  two objectives. First, it enhances concise initial scene descriptions, evolving
  them into detailed forms while dynamically adapting the text based on
  subsequent instructions. Second, it integrates procedural generation,
  extracting parameter values from enriched text to effortlessly interface with
  3D software for asset creation. Our empirical investigations confirm that
  3D-GPT not only interprets and executes instructions, delivering reliable
  results but also collaborates effectively with human designers. Furthermore, it
  seamlessly integrates with Blender, unlocking expanded manipulation
  possibilities. Our work highlights the potential of LLMs in 3D modeling,
  offering a basic framework for future advancements in scene generation and
  animation.
paper_authors._id: 65320d35ef70d3555f46f4b6
paper_authors.name: Chunyi Sun
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-20T08:53:05.682Z
paper_authors._id: 646dbb124ad7f907279dd18b
paper_authors.avatarUrl: /avatars/b92e76ad7e1e9afdc6fe352584633cea.svg
paper_authors.fullname: sun
paper_authors.type: user
paper_authors.user: chunyin
paper_authors._id: 65320d35ef70d3555f46f4b7
paper_authors.name: Junlin Han
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-20T08:53:12.541Z
paper_authors._id: 636e6ee287545ca5a136b4c3
paper_authors.avatarUrl: /avatars/208d32b1202e2da210146027212dbdd3.svg
paper_authors.fullname: Junlin Han
paper_authors.type: user
paper_authors.user: Junlinh
paper_authors._id: 65320d35ef70d3555f46f4b8
paper_authors.name: Weijian Deng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-20T08:53:19.882Z
paper_authors._id: 6431852ddd466752c73d7639
paper_authors.avatarUrl: /avatars/86d897afdba81bc45ec3946f622e9dcb.svg
paper_authors.fullname: weijian Deng
paper_authors.type: user
paper_authors.user: laonafahaodange
paper_authors._id: 65320d35ef70d3555f46f4b9
paper_authors.name: Xinlong Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-20T08:53:26.662Z
paper_authors._id: 63ca558304c979828311c5a5
paper_authors.avatarUrl: /avatars/2a439d79fba2f987cabe780d10c94d25.svg
paper_authors.fullname: Xinlong Wang
paper_authors.type: user
paper_authors.user: xinlongwang
paper_authors._id: 65320d35ef70d3555f46f4ba
paper_authors.name: Zishan Qin
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-10-24T07:30:00.761Z
paper_authors._id: 6537435553b7eb25cef80f82
paper_authors.avatarUrl: /avatars/16a3c68d58b946f51d73f061a9b7cb85.svg
paper_authors.fullname: Zishan Qin
paper_authors.type: user
paper_authors.user: taylorqin
paper_authors._id: 65320d35ef70d3555f46f4bb
paper_authors.name: Stephen Gould
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-20T08:53:42.305Z
paper_authors._id: 63114d9c1f7088709e49b44c
paper_authors.avatarUrl: /avatars/8515edaef9c3f46ccd4c1e2efa763c98.svg
paper_authors.fullname: Stephen Gould
paper_authors.type: user
paper_authors.user: sgould
paper_title: 3D-GPT: Procedural 3D Modeling with Large Language Models
paper_summary: In the pursuit of efficient automated content creation, procedural
  generation, leveraging modifiable parameters and rule-based systems, emerges as
  a promising approach. Nonetheless, it could be a demanding endeavor, given its
  intricate nature necessitating a deep understanding of rules, algorithms, and
  parameters. To reduce workload, we introduce 3D-GPT, a framework utilizing
  large language models~(LLMs) for instruction-driven 3D modeling. 3D-GPT
  positions LLMs as proficient problem solvers, dissecting the procedural 3D
  modeling tasks into accessible segments and appointing the apt agent for each
  task. 3D-GPT integrates three core agents: the task dispatch agent, the
  conceptualization agent, and the modeling agent. They collaboratively achieve
  two objectives. First, it enhances concise initial scene descriptions, evolving
  them into detailed forms while dynamically adapting the text based on
  subsequent instructions. Second, it integrates procedural generation,
  extracting parameter values from enriched text to effortlessly interface with
  3D software for asset creation. Our empirical investigations confirm that
  3D-GPT not only interprets and executes instructions, delivering reliable
  results but also collaborates effectively with human designers. Furthermore, it
  seamlessly integrates with Blender, unlocking expanded manipulation
  possibilities. Our work highlights the potential of LLMs in 3D modeling,
  offering a basic framework for future advancements in scene generation and
  animation.
paper_ai_keywords: 3D-GPT large language models (LLMs) task dispatch agent conceptualization agent modeling agent instruction-driven 3D modeling procedural 3D modeling Blender