title: Large Language Models as Planning Domain Generators
summary: Developing domain models is one of the few remaining places that require
  manual human labor in AI planning. Thus, in order to make planning more
  accessible, it is desirable to automate the process of domain model generation.
  To this end, we investigate if large language models (LLMs) can be used to
  generate planning domain models from simple textual descriptions. Specifically,
  we introduce a framework for automated evaluation of LLM-generated domains by
  comparing the sets of plans for domain instances. Finally, we perform an
  empirical analysis of 7 large language models, including coding and chat models
  across 9 different planning domains, and under three classes of natural
  language domain descriptions. Our results indicate that LLMs, particularly
  those with high parameter counts, exhibit a moderate level of proficiency in
  generating correct planning domains from natural language descriptions. Our
  code is available at https://github.com/IBM/NL2PDDL.
paper_authors._id: 6642e7a724555ed542ec83fd
paper_authors.name: James Oswald
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2025-02-16T22:58:59.792Z
paper_authors._id: 648b152a4cb9bdfc9fe144aa
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/jaiMWXW9_6AEiDhgbIb7T.png
paper_authors.fullname: James Oswald
paper_authors.type: user
paper_authors.user: James-Oswald
paper_authors._id: 6642e7a724555ed542ec83fe
paper_authors.name: Kavitha Srinivas
paper_authors._id: 6642e7a724555ed542ec83ff
paper_authors.name: Harsha Kokel
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-02T08:57:30.109Z
paper_authors._id: 648b67b2355113a5fba0ff31
paper_authors.avatarUrl: /avatars/8ea58a2e3f261d20b33c033992bc7f52.svg
paper_authors.fullname: Harsha Kokel
paper_authors.type: user
paper_authors.user: harshakokel
paper_authors._id: 6642e7a724555ed542ec8400
paper_authors.name: Junkyu Lee
paper_authors._id: 6642e7a724555ed542ec8401
paper_authors.name: Michael Katz
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-31T15:15:06.023Z
paper_authors._id: 65b2d595b389ca2de1945d59
paper_authors.avatarUrl: /avatars/768bb50d7fd0d3cb416fe63645b3a658.svg
paper_authors.fullname: Michael Katz
paper_authors.type: user
paper_authors.user: ctpelok
paper_authors._id: 6642e7a724555ed542ec8402
paper_authors.name: Shirin Sohrabi
paper_title: Large Language Models as Planning Domain Generators
paper_summary: Developing domain models is one of the few remaining places that require
  manual human labor in AI planning. Thus, in order to make planning more
  accessible, it is desirable to automate the process of domain model generation.
  To this end, we investigate if large language models (LLMs) can be used to
  generate planning domain models from simple textual descriptions. Specifically,
  we introduce a framework for automated evaluation of LLM-generated domains by
  comparing the sets of plans for domain instances. Finally, we perform an
  empirical analysis of 7 large language models, including coding and chat models
  across 9 different planning domains, and under three classes of natural
  language domain descriptions. Our results indicate that LLMs, particularly
  those with high parameter counts, exhibit a moderate level of proficiency in
  generating correct planning domains from natural language descriptions. Our
  code is available at https://github.com/IBM/NL2PDDL.
paper_ai_summary: Large language models show moderate proficiency in generating automated planning domain models from natural language descriptions.
paper_ai_keywords: domain models AI planning large language models (LLMs) automated evaluation planning domain models natural language descriptions