Intelligence at the Edge of Chaos
We explore the emergence of intelligent behavior in artificial systems by
investigating how the complexity of rule-based systems influences the
capabilities of models trained to predict these rules. Our study focuses on
elementary cellular automata (ECA), simple yet powerful one-dimensional systems
that generate behaviors ranging from trivial to highly complex. By training
distinct Large Language Models (LLMs) on different ECAs, we evaluated the
relationship between the complexity of the rules' behavior and the intelligence
exhibited by the LLMs, as reflected in their performance on downstream tasks.
Our findings reveal that rules with higher complexity lead to models exhibiting
greater intelligence, as demonstrated by their performance on reasoning and
chess move prediction tasks. Both uniform and periodic systems, and often also
highly chaotic systems, resulted in poorer downstream performance, highlighting
a sweet spot of complexity conducive to intelligence. We conjecture that
intelligence arises from the ability to predict complexity and that creating
intelligence may require only exposure to complexity.
67001603d78e24f7e25061a2
Shiyang Zhang
67001603d78e24f7e25061a3
Aakash Patel
extracted_confirmed
2024-10-04T16:23:47.544Z
63955025254ed3075fb26d53
/avatars/c84049bbfff80c82b95e4bc76512e2ce.svg
Aakash Patel
user
aakashdp
67001603d78e24f7e25061a4
Syed A Rizvi
67001603d78e24f7e25061a5
Nianchen Liu
67001603d78e24f7e25061a6
Sizhuang He
claimed_verified
2024-11-15T09:43:57.931Z
652427b506e839c554fec500
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/-FGPTca35zl2i79ubPcjV.png
Sizhuang He
user
sizhuang
67001603d78e24f7e25061a7
Amin Karbasi
67001603d78e24f7e25061a8
Emanuele Zappala
67001603d78e24f7e25061a9
David van Dijk
extracted_pending
2024-10-04T16:21:25.022Z
6514a30dfe79293513eeea09
/avatars/bdc0a6338872568274c759fd1813fd63.svg
David van Dijk
user
davidvandijk
Intelligence at the Edge of Chaos
We explore the emergence of intelligent behavior in artificial systems by
investigating how the complexity of rule-based systems influences the
capabilities of models trained to predict these rules. Our study focuses on
elementary cellular automata (ECA), simple yet powerful one-dimensional systems
that generate behaviors ranging from trivial to highly complex. By training
distinct Large Language Models (LLMs) on different ECAs, we evaluated the
relationship between the complexity of the rules' behavior and the intelligence
exhibited by the LLMs, as reflected in their performance on downstream tasks.
Our findings reveal that rules with higher complexity lead to models exhibiting
greater intelligence, as demonstrated by their performance on reasoning and
chess move prediction tasks. Both uniform and periodic systems, and often also
highly chaotic systems, resulted in poorer downstream performance, highlighting
a sweet spot of complexity conducive to intelligence. We conjecture that
intelligence arises from the ability to predict complexity and that creating
intelligence may require only exposure to complexity.
Training LLMs on rules from elementary cellular automata reveals that moderate complexity in the rules enhances the models' intelligence and performance on reasoning and chess tasks.
Large Language Models LLCs elementary cellular automata ECA complexity reasoning chess move prediction uniform systems periodic systems chaotic systems