title: Intelligence at the Edge of Chaos
summary: We explore the emergence of intelligent behavior in artificial systems by
  investigating how the complexity of rule-based systems influences the
  capabilities of models trained to predict these rules. Our study focuses on
  elementary cellular automata (ECA), simple yet powerful one-dimensional systems
  that generate behaviors ranging from trivial to highly complex. By training
  distinct Large Language Models (LLMs) on different ECAs, we evaluated the
  relationship between the complexity of the rules' behavior and the intelligence
  exhibited by the LLMs, as reflected in their performance on downstream tasks.
  Our findings reveal that rules with higher complexity lead to models exhibiting
  greater intelligence, as demonstrated by their performance on reasoning and
  chess move prediction tasks. Both uniform and periodic systems, and often also
  highly chaotic systems, resulted in poorer downstream performance, highlighting
  a sweet spot of complexity conducive to intelligence. We conjecture that
  intelligence arises from the ability to predict complexity and that creating
  intelligence may require only exposure to complexity.
paper_authors._id: 67001603d78e24f7e25061a2
paper_authors.name: Shiyang Zhang
paper_authors._id: 67001603d78e24f7e25061a3
paper_authors.name: Aakash Patel
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2024-10-04T16:23:47.544Z
paper_authors._id: 63955025254ed3075fb26d53
paper_authors.avatarUrl: /avatars/c84049bbfff80c82b95e4bc76512e2ce.svg
paper_authors.fullname: Aakash Patel
paper_authors.type: user
paper_authors.user: aakashdp
paper_authors._id: 67001603d78e24f7e25061a4
paper_authors.name: Syed A Rizvi
paper_authors._id: 67001603d78e24f7e25061a5
paper_authors.name: Nianchen Liu
paper_authors._id: 67001603d78e24f7e25061a6
paper_authors.name: Sizhuang He
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-15T09:43:57.931Z
paper_authors._id: 652427b506e839c554fec500
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/-FGPTca35zl2i79ubPcjV.png
paper_authors.fullname: Sizhuang He
paper_authors.type: user
paper_authors.user: sizhuang
paper_authors._id: 67001603d78e24f7e25061a7
paper_authors.name: Amin Karbasi
paper_authors._id: 67001603d78e24f7e25061a8
paper_authors.name: Emanuele Zappala
paper_authors._id: 67001603d78e24f7e25061a9
paper_authors.name: David van Dijk
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2024-10-04T16:21:25.022Z
paper_authors._id: 6514a30dfe79293513eeea09
paper_authors.avatarUrl: /avatars/bdc0a6338872568274c759fd1813fd63.svg
paper_authors.fullname: David van Dijk
paper_authors.type: user
paper_authors.user: davidvandijk
paper_title: Intelligence at the Edge of Chaos
paper_summary: We explore the emergence of intelligent behavior in artificial systems by
  investigating how the complexity of rule-based systems influences the
  capabilities of models trained to predict these rules. Our study focuses on
  elementary cellular automata (ECA), simple yet powerful one-dimensional systems
  that generate behaviors ranging from trivial to highly complex. By training
  distinct Large Language Models (LLMs) on different ECAs, we evaluated the
  relationship between the complexity of the rules' behavior and the intelligence
  exhibited by the LLMs, as reflected in their performance on downstream tasks.
  Our findings reveal that rules with higher complexity lead to models exhibiting
  greater intelligence, as demonstrated by their performance on reasoning and
  chess move prediction tasks. Both uniform and periodic systems, and often also
  highly chaotic systems, resulted in poorer downstream performance, highlighting
  a sweet spot of complexity conducive to intelligence. We conjecture that
  intelligence arises from the ability to predict complexity and that creating
  intelligence may require only exposure to complexity.
paper_ai_summary: Training LLMs on rules from elementary cellular automata reveals that moderate complexity in the rules enhances the models' intelligence and performance on reasoning and chess tasks.
paper_ai_keywords: Large Language Models LLCs elementary cellular automata ECA complexity reasoning chess move prediction uniform systems periodic systems chaotic systems