title: Sibyl: Simple yet Effective Agent Framework for Complex Real-world
    Reasoning
summary: Existing agents based on large language models (LLMs) demonstrate robust
  problem-solving capabilities by integrating LLMs' inherent knowledge, strong
  in-context learning and zero-shot capabilities, and the use of tools combined
  with intricately designed LLM invocation workflows by humans. However, these
  agents still exhibit shortcomings in long-term reasoning and under-use the
  potential of existing tools, leading to noticeable deficiencies in complex
  real-world reasoning scenarios. To address these limitations, we introduce
  Sibyl, a simple yet powerful LLM-based agent framework designed to tackle
  complex reasoning tasks by efficiently leveraging a minimal set of tools.
  Drawing inspiration from Global Workspace Theory, Sibyl incorporates a global
  workspace to enhance the management and sharing of knowledge and conversation
  history throughout the system. Furthermore, guided by Society of Mind Theory,
  Sibyl implements a multi-agent debate-based jury to self-refine the final
  answers, ensuring a comprehensive and balanced approach. This approach aims to
  reduce system complexity while expanding the scope of problems solvable-from
  matters typically resolved by humans in minutes to those requiring hours or
  even days, thus facilitating a shift from System-1 to System-2 thinking. Sibyl
  has been designed with a focus on scalability and ease of debugging by
  incorporating the concept of reentrancy from functional programming from its
  inception, with the aim of seamless and low effort integration in other LLM
  applications to improve capabilities. Our experimental results on the GAIA
  benchmark test set reveal that the Sibyl agent instantiated with GPT-4 achieves
  state-of-the-art performance with an average score of 34.55%, compared to other
  agents based on GPT-4. We hope that Sibyl can inspire more reliable and
  reusable LLM-based agent solutions to address complex real-world reasoning
  tasks.
paper_authors._id: 6695f0dc22739e087e9f60a7
paper_authors.name: Yulong Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-07-16T20:15:31.748Z
paper_authors._id: 60ec4a33375c1280fb422704
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1633664276374-60ec4a33375c1280fb422704.png
paper_authors.fullname: Wang Yulong
paper_authors.type: user
paper_authors.user: wangyulong
paper_authors._id: 6695f0dc22739e087e9f60a8
paper_authors.name: Tianhao Shen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-04T17:06:51.969Z
paper_authors._id: 61976063b1099a2392101960
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/61976063b1099a2392101960/DRlAoI6cD6yyomQfWRTU5.png
paper_authors.fullname: Tianhao Shen
paper_authors.type: user
paper_authors.user: magnetic
paper_authors._id: 6695f0dc22739e087e9f60a9
paper_authors.name: Lifeng Liu
paper_authors._id: 6695f0dc22739e087e9f60aa
paper_authors.name: Jian Xie
paper_title: Sibyl: Simple yet Effective Agent Framework for Complex Real-world
    Reasoning
paper_summary: Existing agents based on large language models (LLMs) demonstrate robust
  problem-solving capabilities by integrating LLMs' inherent knowledge, strong
  in-context learning and zero-shot capabilities, and the use of tools combined
  with intricately designed LLM invocation workflows by humans. However, these
  agents still exhibit shortcomings in long-term reasoning and under-use the
  potential of existing tools, leading to noticeable deficiencies in complex
  real-world reasoning scenarios. To address these limitations, we introduce
  Sibyl, a simple yet powerful LLM-based agent framework designed to tackle
  complex reasoning tasks by efficiently leveraging a minimal set of tools.
  Drawing inspiration from Global Workspace Theory, Sibyl incorporates a global
  workspace to enhance the management and sharing of knowledge and conversation
  history throughout the system. Furthermore, guided by Society of Mind Theory,
  Sibyl implements a multi-agent debate-based jury to self-refine the final
  answers, ensuring a comprehensive and balanced approach. This approach aims to
  reduce system complexity while expanding the scope of problems solvable-from
  matters typically resolved by humans in minutes to those requiring hours or
  even days, thus facilitating a shift from System-1 to System-2 thinking. Sibyl
  has been designed with a focus on scalability and ease of debugging by
  incorporating the concept of reentrancy from functional programming from its
  inception, with the aim of seamless and low effort integration in other LLM
  applications to improve capabilities. Our experimental results on the GAIA
  benchmark test set reveal that the Sibyl agent instantiated with GPT-4 achieves
  state-of-the-art performance with an average score of 34.55%, compared to other
  agents based on GPT-4. We hope that Sibyl can inspire more reliable and
  reusable LLM-based agent solutions to address complex real-world reasoning
  tasks.
paper_ai_keywords: large language models (LLMs) in-context learning zero-shot capabilities long-term reasoning global workspace Global Workspace Theory multi-agent debate-based jury Society of Mind Theory System-1 thinking System-2 thinking scalability reentrancy functional programming GAIA benchmark GPT-4