title: LLM Agent Operating System
summary: The integration and deployment of large language model (LLM)-based
  intelligent agents have been fraught with challenges that compromise their
  efficiency and efficacy. Among these issues are sub-optimal scheduling and
  resource allocation of agent requests over the LLM, the difficulties in
  maintaining context during interactions between agent and LLM, and the
  complexities inherent in integrating heterogeneous agents with different
  capabilities and specializations. The rapid increase of agent quantity and
  complexity further exacerbates these issues, often leading to bottlenecks and
  sub-optimal utilization of resources. Inspired by these challenges, this paper
  presents AIOS, an LLM agent operating system, which embeds large language model
  into operating systems (OS). Specifically, AIOS is designed to optimize
  resource allocation, facilitate context switch across agents, enable concurrent
  execution of agents, provide tool service for agents, and maintain access
  control for agents. We present the architecture of such an operating system,
  outline the core challenges it aims to resolve, and provide the basic design
  and implementation of the AIOS. Our experiments on concurrent execution of
  multiple agents demonstrate the reliability and efficiency of our AIOS modules.
  Through this, we aim to not only improve the performance and efficiency of LLM
  agents but also to pioneer for better development and deployment of the AIOS
  ecosystem in the future. The project is open-source at
  https://github.com/agiresearch/AIOS.
paper_authors._id: 660245ac727c75cf509100bf
paper_authors.name: Kai Mei
paper_authors._id: 660245ac727c75cf509100c0
paper_authors.name: Zelong Li
paper_authors._id: 660245ac727c75cf509100c1
paper_authors.name: Shuyuan Xu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-26T08:37:15.496Z
paper_authors._id: 64f3aee61b9aa0ded19651dd
paper_authors.avatarUrl: /avatars/f60d9ead11e3a30e0a328dd04f5637b3.svg
paper_authors.fullname: Shuyuan Xu
paper_authors.type: user
paper_authors.user: SeanSX
paper_authors._id: 660245ac727c75cf509100c2
paper_authors.name: Ruosong Ye
paper_authors._id: 660245ac727c75cf509100c3
paper_authors.name: Yingqiang Ge
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-26T08:37:45.032Z
paper_authors._id: 631ce034a31a0a462ca26e3d
paper_authors.avatarUrl: /avatars/5ca70e2d05f826905485654eb232fd16.svg
paper_authors.fullname: Yingqiang Ge
paper_authors.type: user
paper_authors.user: gyq1101
paper_authors._id: 660245ac727c75cf509100c4
paper_authors.name: Yongfeng Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-26T08:37:51.545Z
paper_authors._id: 63b0deb7f2eb87a4d695055c
paper_authors.avatarUrl: /avatars/85a87fa82f7579019a0c6212f03b6480.svg
paper_authors.fullname: Yongfeng Zhang
paper_authors.type: user
paper_authors.user: evison
paper_title: LLM Agent Operating System
paper_summary: The integration and deployment of large language model (LLM)-based
  intelligent agents have been fraught with challenges that compromise their
  efficiency and efficacy. Among these issues are sub-optimal scheduling and
  resource allocation of agent requests over the LLM, the difficulties in
  maintaining context during interactions between agent and LLM, and the
  complexities inherent in integrating heterogeneous agents with different
  capabilities and specializations. The rapid increase of agent quantity and
  complexity further exacerbates these issues, often leading to bottlenecks and
  sub-optimal utilization of resources. Inspired by these challenges, this paper
  presents AIOS, an LLM agent operating system, which embeds large language model
  into operating systems (OS). Specifically, AIOS is designed to optimize
  resource allocation, facilitate context switch across agents, enable concurrent
  execution of agents, provide tool service for agents, and maintain access
  control for agents. We present the architecture of such an operating system,
  outline the core challenges it aims to resolve, and provide the basic design
  and implementation of the AIOS. Our experiments on concurrent execution of
  multiple agents demonstrate the reliability and efficiency of our AIOS modules.
  Through this, we aim to not only improve the performance and efficiency of LLM
  agents but also to pioneer for better development and deployment of the AIOS
  ecosystem in the future. The project is open-source at
  https://github.com/agiresearch/AIOS.
paper_ai_summary: AIOS, an operating system embedding large language models, addresses resource allocation, context switching, and concurrency challenges for intelligent agents, demonstrating reliability and efficiency.
paper_ai_keywords: large language model intelligent agents AIOS resource allocation context switch concurrent execution access control