From an LLM Swarm to a PDDL-empowered Hive: Planning Self-executed Instructions in a Multi-modal Jungle
Kaustubh Vyas Damien Graux Yijun Yang Sebastien Montella Chenxin Diao Wendi Zhou Pavlos Vougiouklis Ruofei Lai Yang Ren Keshuang Li Jeff Z. Pan
In response to the call for agent-based solutions that leverage the ever-increasing capabilities of the deep models' ecosystem, we introduce a comprehensive solution for selecting appropriate models and subsequently planning a set of atomic actions to satisfy the end-users' instructions.

Our system, Hive, operates over sets of models and, upon receiving natural language instructions, schedules and executes, explainable plans of atomic actions. These actions can involve one or more of the available models to achieve the overall task, while respecting end-users specific constraints. Hive is able to plan complex chains of actions while guaranteeing explainability, using an LLM-based formal logic backbone empowered by PDDL operations. We introduce the MuSE benchmark in order to offer a comprehensive evaluation of the multi-modal capabilities of agent systems. Our findings show that our framework redefines the state-of-the-art for task selection, outperforming other competing systems that plan operations across multiple models while offering transparency guarantees while fully adhering to user constraints.
Deep Models, Planning, PDDL, Knowledge Graphs, Benchmark, Large Language Models
Introducing Hive: a powerful, explainable system for selecting models & planning atomic actions based on natural language instructions. Hive leverages PDDL to deliver complex multi-modal tasks while respecting user constraints.