Exploring and Benchmarking the Planning Capabilities of Large Language Models
<|reference_start|>Exploring and Benchmarking the Planning Capabilities of Large Language Models: We seek to elevate the planning capabilities of Large Language Models (LLMs)investigating four main directions. First, we construct a comprehensive benchmark suite encompassing both classical planning domains and natural language scenarios. This suite includes algorithms to generate instances with varying levels of difficulty, allowing for rigorous and systematic evaluation of LLM performance. Second, we investigate the use of in-context learning (ICL) to enhance LLM planning, exploring the direct relationship between increased context length and improved planning performance. Third, we demonstrate the positive impact of fine-tuning LLMs on optimal planning paths, as well as the effectiveness of incorporating model-driven search procedures. Finally, we investigate the performance of the proposed methods in out-of-distribution scenarios, assessing the ability to generalize to novel and unseen planning challenges.<|reference_end|>
@article{bohnet2024exploring,
    title={Exploring and Benchmarking the Planning Capabilities of Large Language
  Models},
    author={Bernd Bohnet, Azade Nova, Aaron T Parisi, Kevin Swersky, Katayoon
  Goshvadi, Hanjun Dai, Dale Schuurmans, Noah Fiedel, Hanie Sedghi},
    journal={arXiv preprint arXiv:2406.13094},
    year={2024},
    archivePrefix={arXiv},
    eprint={2406.13094},
    primaryClass={cs.CL cs.AI cs.LG}
}