title: Pangu-Agent: A Fine-Tunable Generalist Agent with Structured Reasoning
summary: A key method for creating Artificial Intelligence (AI) agents is
  Reinforcement Learning (RL). However, constructing a standalone RL policy that
  maps perception to action directly encounters severe problems, chief among them
  being its lack of generality across multiple tasks and the need for a large
  amount of training data. The leading cause is that it cannot effectively
  integrate prior information into the perception-action cycle when devising the
  policy. Large language models (LLMs) emerged as a fundamental way to
  incorporate cross-domain knowledge into AI agents but lack crucial learning and
  adaptation toward specific decision problems. This paper presents a general
  framework model for integrating and learning structured reasoning into AI
  agents' policies. Our methodology is motivated by the modularity found in the
  human brain. The framework utilises the construction of intrinsic and extrinsic
  functions to add previous understandings of reasoning structures. It also
  provides the adaptive ability to learn models inside every module or function,
  consistent with the modular structure of cognitive processes. We describe the
  framework in-depth and compare it with other AI pipelines and existing
  frameworks. The paper explores practical applications, covering experiments
  that show the effectiveness of our method. Our results indicate that AI agents
  perform and adapt far better when organised reasoning and prior knowledge are
  embedded. This opens the door to more resilient and general AI agent systems.
paper_authors._id: 658a4c122021ba68d7d154d6
paper_authors.name: Filippos Christianos
paper_authors._id: 658a4c122021ba68d7d154d7
paper_authors.name: Georgios Papoudakis
paper_authors._id: 658a4c122021ba68d7d154d8
paper_authors.name: Matthieu Zimmer
paper_authors._id: 658a4c122021ba68d7d154d9
paper_authors.name: Thomas Coste
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-03-14T12:31:47.889Z
paper_authors._id: 63bbef5e141c7d395c468630
paper_authors.avatarUrl: /avatars/1c35c493798fa937fd5502a31482a4d7.svg
paper_authors.fullname: Thomas
paper_authors.type: user
paper_authors.user: tlc4418
paper_authors._id: 658a4c122021ba68d7d154da
paper_authors.name: Zhihao Wu
paper_authors._id: 658a4c122021ba68d7d154db
paper_authors.name: Jingxuan Chen
paper_authors._id: 658a4c122021ba68d7d154dc
paper_authors.name: Khyati Khandelwal
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-01-04T09:34:31.398Z
paper_authors._id: 64671c2d76bb704aa13f83db
paper_authors.avatarUrl: /avatars/e053199e8758bee3f10d0dea1cca2e7f.svg
paper_authors.fullname: Khyati Khandelwal
paper_authors.type: user
paper_authors.user: khyatikhandelwal
paper_authors._id: 658a4c122021ba68d7d154dd
paper_authors.name: James Doran
paper_authors._id: 658a4c122021ba68d7d154de
paper_authors.name: Xidong Feng
paper_authors._id: 658a4c122021ba68d7d154df
paper_authors.name: Jiacheng Liu
paper_authors._id: 658a4c122021ba68d7d154e0
paper_authors.name: Zheng Xiong
paper_authors._id: 658a4c122021ba68d7d154e1
paper_authors.name: Yicheng Luo
paper_authors._id: 658a4c122021ba68d7d154e2
paper_authors.name: Jianye Hao
paper_authors._id: 658a4c122021ba68d7d154e3
paper_authors.name: Kun Shao
paper_authors._id: 658a4c122021ba68d7d154e4
paper_authors.name: Haitham Bou-Ammar
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-01-10T08:39:03.833Z
paper_authors._id: 631c375768f7da9ad2496bf6
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/631c375768f7da9ad2496bf6/1sDOoecA6e1v_hn_VAgUq.jpeg
paper_authors.fullname: Haitham Bou Ammar
paper_authors.type: user
paper_authors.user: hba123
paper_authors._id: 658a4c122021ba68d7d154e5
paper_authors.name: Jun Wang
paper_title: Pangu-Agent: A Fine-Tunable Generalist Agent with Structured Reasoning
paper_summary: A key method for creating Artificial Intelligence (AI) agents is
  Reinforcement Learning (RL). However, constructing a standalone RL policy that
  maps perception to action directly encounters severe problems, chief among them
  being its lack of generality across multiple tasks and the need for a large
  amount of training data. The leading cause is that it cannot effectively
  integrate prior information into the perception-action cycle when devising the
  policy. Large language models (LLMs) emerged as a fundamental way to
  incorporate cross-domain knowledge into AI agents but lack crucial learning and
  adaptation toward specific decision problems. This paper presents a general
  framework model for integrating and learning structured reasoning into AI
  agents' policies. Our methodology is motivated by the modularity found in the
  human brain. The framework utilises the construction of intrinsic and extrinsic
  functions to add previous understandings of reasoning structures. It also
  provides the adaptive ability to learn models inside every module or function,
  consistent with the modular structure of cognitive processes. We describe the
  framework in-depth and compare it with other AI pipelines and existing
  frameworks. The paper explores practical applications, covering experiments
  that show the effectiveness of our method. Our results indicate that AI agents
  perform and adapt far better when organised reasoning and prior knowledge are
  embedded. This opens the door to more resilient and general AI agent systems.
paper_ai_keywords: Reinforcement Learning (RL) perception-action cycle Large language models (LLMs) intrinsic functions extrinsic functions structured reasoning cognitive processes modularity adaptive ability organisational reasoning