summaries: Liquid State Machines (LSM)  circumvents the need for backpropagation and allows the training of recurrent neural networks by local plasticity in the readout weights. Previous studies have shown that optimal performance is achieved near a critical phase transition of the reservoir (the liquid). However, the exact parameters of the critical phase transition in the liquid vary and depend on the input. In this work, the authors present a biologically inspired model that allows self-organization near the critical transition. Here, the authors propose a biologically inspired model in which the liquid self-organizes near the critical point. The model is composed of a reservoir of integrate-and-fire neurons. The synaptic efficacies within have unsupervised spike-time dependent plasticity (STDP). The authors propose an additional dynamical variable akin to the activity of astrocytes in the circuit, which modifies the STDP learning rates. The model self-organizes close to the critical point. The authors train several networks on the MNIST and N-MNIST dataset (a spiking version of MNIST, made for neuromorphic computing). They show that the SLM model with astrocyte-dependent STDP plasticity outperforms other LSM models. The authors propose of a way modulating the connections in a liquid state machine (LSM) so that it moves towards the edge of chaos (EOC) to maximize its performance for various tasks. They do this using by introducing an additional "astrocyte" that modulates the parameters of STDP in the neuron model -- specifically the depression term. The authors compare their model to other forms of LSMs.  This paper extends liquid state machine (LSM) with a biologically inspired astrocyte model (NALSM) to optimize the neuronal dynamics in LSM at the edge-of-chaos, to improve the accuracy and stability of LSMs. With NALSM, the spiking model can maintain a high accuracy without re-tuning parameters for different datasets. Experiments with MNIST and N-MNIST dataset demonstrate that NALSM outperforms classic LSM and other STDP-based LSM approaches, and most existing spiking neuron models including multi-layer SNNs. This paper presents a novel type of liquid state machine - neuron-astrocyte liquid state machine (NALSM), which introduces feedback from the network activity to the synaptic plasticity. The proposed mechanism helps to position the liquid (reservoir) layer in the edge-of-chaos dynamics regime, a known key factor to achieve good computational performance. The main contribution of this paper is transforming a biological observation (the role played by astrocytes for modulating the synaptic plasticity) into a novel idea of modulating the liquid activity. It uses feedbacks to obtain edge-of-chaos property in liquid state machine (LSM), and further showing meaningful improvement over the previously proposed LSM-based methods for real-world tasks. The paper proposes a new version of LSM by adding a factor intended to mimic a proposed role for astrocytes: modulating STDP. The effect is homeostatic in nature and works to keep the network in a computationally efficient regime. It works. The result is a higher performance on MNIST and N-MNIST data sets. 
abstractText: The liquid state machine (LSM) combines low training complexity and biological plausibility, which has made it an attractive machine learning framework for edge and neuromorphic computing paradigms. Originally proposed as a model of brain computation, the LSM tunes its internal weights without backpropagation of gradients, which results in lower performance compared to multi-layer neural networks. Recent findings in neuroscience suggest that astrocytes, a long-neglected non-neuronal brain cell, modulate synaptic plasticity and brain dynamics, tuning brain networks to the vicinity of the computationally optimal critical phase transition between order and chaos. Inspired by this disruptive understanding of how brain networks self-tune, we propose the neuron-astrocyte liquid state machine (NALSM)1 that addresses under-performance through self-organized near-critical dynamics. Similar to its biological counterpart, the astrocyte model integrates neuronal activity and provides global feedback to spike-timing-dependent plasticity (STDP), which self-organizes NALSM dynamics around a critical branching factor that is associated with the edge-of-chaos. We demonstrate that NALSM achieves state-of-the-art accuracy versus comparable LSM methods, without the need for data-specific hand-tuning. With a top accuracy of 97.61% on MNIST, 97.51% on N-MNIST, and 85.84% on Fashion-MNIST, NALSM achieved comparable performance to current fully-connected multi-layer spiking neural networks trained via backpropagation. Our findings suggest that the further development of braininspired machine learning methods has the potential to reach the performance of deep learning, with the added benefits of supporting robust and energy-efficient neuromorphic computing on the edge.
authors.name: Vladimir A. Ivanov
authors.name: Konstantinos P. Michmizos
references.authors: Jack D. Kendall Suhas Kumar
references.title: The building blocks of a brain-inspired computer
references.venue: Applied Physics Reviews,
references.authors: Guangzhi Tang Neelesh Kumar Konstantinos P. Michmizos
references.title: Reinforcement co-learning of deep and spiking neural networks for energy-efficient mapless navigation with neuromorphic hardware
references.venue: In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS),
references.authors: Mike Davies Narayan Srinivasa Tsung-Han Lin Gautham Chinya Yongqiang Cao Sri Harsha Choday Georgios Dimou Prasad Joshi Nabil Imam Shweta Jain Yuyun Liao Chit-Kwan Lin Andrew Lines Ruokun Liu Deepak Mathaikutty Steven McCoy Arnab Paul Jonathan Tse Guruguhanathan Venkataramanan Yi-Hsin Weng Andreas Wild Yoonseok Yang Hong Wang
references.title: Loihi: A neuromorphic manycore processor with on-chip learning
references.venue: IEEE Micro,
references.authors: Guangzhi Tang Arpit Shah Konstantinos P. Michmizos
references.title: Spiking neural network on neuromorphic hardware for energy-efficient unidimensional slam
references.venue: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS),
references.authors: Keyan Cao Yefan Liu Gongjie Meng Qimeng Sun
references.title: An overview on edge computing research
references.venue: IEEE Access,
references.authors: Jiasi Chen Xukan Ran
references.title: Deep learning with edge computing: A review
references.venue: Proceedings of the IEEE,
references.authors: Wolfgang Maass Thomas Natschläger Henry Markram
references.title: Real-time computing without stable states: A new framework for neural computation based on perturbations
references.venue: Neural Computation,
references.title: Deep liquid state machines with neural plasticity for video activity recognition
references.venue: Frontiers in Neuroscience,
references.authors: Wachirawit Ponghiran Gopalakrishnan Srinivasan Kaushik Roy
references.title: Reinforcement learning with low-complexity liquid state machines
references.venue: Frontiers in Neuroscience,
references.authors: Qian Wang Peng Li
references.title: D-lsm: Deep liquid state machine with unsupervised recurrent reservoir tuning
references.venue: In 2016 23rd International Conference on Pattern Recognition (ICPR),
references.authors: Shiya Liu Lingjia Liu Yang Yi
references.title: Quantized reservoir computing on edge devices for communication applications
references.venue: In 2020 IEEE/ACM Symposium on Edge Computing (SEC),
references.authors: Shiming Li Lei Wang Shiying Wang Weixia Xu
references.title: Liquid state machine applications mapping for noc-based neuromorphic platforms
references.venue: Advanced Computer Architecture,
references.authors: Bon Woong Ku Yu Liu Yingyezhe Jin Sandeep Samal Peng Li Sung Kyu Lim
references.title: Design and architectural co-optimization of monolithic 3d liquid state machine-based neuromorphic processor
references.venue: 55th ACM/ESDA/IEEE Design Automation Conference (DAC),
references.authors: Josep L. Rosselló Miquel L. Alomar Antoni Morro Antoni Oliver Vincent Canals
references.title: Highdensity liquid-state machine circuitry for time-series forecasting
references.venue: International Journal of Neural Systems,
references.authors: Ismael Balafrej Jean Rouat
references.title: P-critical: A reservoir autoregulation plasticity rule for neuromorphic hardware, 2020
references.authors: Simon Brodeur Jean Rouat
references.title: Regulation toward self-organized criticality in a recurrent spiking neural reservoir
references.venue: Artificial Neural Networks and Machine Learning – ICANN
references.authors: Yingyezhe Jin Peng Li
references.title: Ap-stdp: A novel self-organizing mechanism for efficient reservoir computing
references.venue: In 2016 International Joint Conference on Neural Networks (IJCNN),
references.authors: D. Norton D. Ventura
references.title: Preparing more effective liquid state machines using hebbian learning
references.venue: IEEE International Joint Conference on Neural Network Proceedings,
references.authors: Robert Legenstein Wolfgang Maass
references.title: Edge of chaos and prediction of computational performance for neural circuit models
references.venue: Neural Networks,
references.authors: Nils Bertschinger Thomas Natschläger
references.title: Real-time computation at the edge of chaos in recurrent neural networks
references.venue: Neural Computation,
references.authors: Chris G. Langton
references.title: Computation at the edge of chaos: phase transitions and emergent computation
references.venue: Physica D: Nonlinear Phenomena,
references.authors: Woodrow L. Shew Hongdian Yang Shan Yu Rajarshi Roy Dietmar Plenz
references.title: Information capacity and transmission are maximized in balanced cortical networks with neuronal avalanches
references.venue: Journal of Neuroscience,
references.authors: Lucilla de Arcangelis Hans J. Herrmann
references.title: Learning as a phenomenon occurring in a critical state
references.venue: Proceedings of the National Academy of Sciences,
references.authors: Osame Kinouchi Mauro Copelli
references.title: Optimal dynamical range of excitable networks at criticality
references.venue: Nature Physics,
references.authors: Clayton Haldeman John M. Beggs
references.title: Critical branching captures activity in living neural networks and maximizes the number of metastable states
references.venue: Phys. Rev. Lett.,
references.authors: John M. Beggs Dietmar Plenz
references.title: Neuronal avalanches in neocortical circuits
references.venue: Journal of Neuroscience,
references.authors: Oren Shriki Jeff Alstott Frederick Carver Tom Holroyd Richard N.A. Henson Marie L. Smith Richard Coppola Edward Bullmore Dietmar Plenz
references.title: Neuronal avalanches in the resting meg of the human brain
references.venue: Journal of Neuroscience,
references.authors: Dante R. Chialvo
references.title: Emergent complex neural dynamics
references.venue: Nature Physics,
references.authors: Daniel E. Feldman
references.title: Synaptic mechanisms for plasticity in neocortex
references.venue: Annual Review of Neuroscience,
references.authors: Nigel Stepp Dietmar Plenz Narayan Srinivasa
references.title: Synaptic plasticity enables adaptive selftuning critical networks
references.venue: PLOS Computational Biology, 11(1):1–28,
references.authors: Friedemann Zenke Wulfram Gerstner
references.title: Hebbian plasticity requires compensatory processes on multiple timescales
references.venue: Philosophical Transactions of the Royal Society B: Biological Sciences,
references.authors: Alanna Watt Niraj Desai
references.title: Homeostatic plasticity and stdp: keeping a neuron’s cool in a fluctuating world
references.venue: Frontiers in Synaptic Neuroscience,
references.authors: L.F. Abbott Sacha B. Nelson
references.title: Synaptic plasticity: taming the beast
references.venue: Nature Neuroscience,
references.authors: Gertrudis Perea Alfonso Araque
references.title: Glia modulates synaptic transmission
references.venue: Brain Research Reviews,
references.authors: Xiaoning Han Michael Chen Fushun Wang Martha Windrem Su Wang Steven Shanz Qiwu Xu Nancy Ann Oberheim Lane Bekar Sarah Betstadt
references.title: Forebrain engraftment by human glial progenitor cells enhances synaptic plasticity and learning in adult mice
references.venue: Cell Stem Cell,
references.authors: Shivendra Tewari Vladimir Parpura
references.title: A possible role of astrocytes in contextual memory retrieval: An analysis obtained using a quantitative framework
references.venue: Frontiers in Computational Neuroscience,
references.authors: Marta Navarrete Alfonso Araque
references.title: Endocannabinoids potentiate synaptic transmission through stimulation of astrocytes
references.venue: Neuron, 68(1):113–126,
references.authors: Adar Adamsky Adi Kol Tirzah Kreisel Adi Doron Nofar Ozeri-Engelhard Talia Melcer Ron Refaeli Henrike Horn Limor Regev Maya Groysman
references.title: Astrocytic activation generates de novo neuronal potentiation and memory enhancement
references.venue: Cell, 174(1):59–71
references.authors: Francesco Petrelli Glenn Dallérac Luca Pucci Corrado Calì Tamara Zehnder Sébastien Sultan Salvatore Lecca Andrea Chicca Andrei Ivanov Cédric S. Asensio Vidar Gundersen Nicolas Toni Graham William Knott Fulvio Magara Jürg Gertsch Frank Kirchhoff Nicole Déglon Bruno Giros Robert H. Edwards Jean-Pierre Mothet Paola Bezzi
references.title: Dysfunction of homeostatic control of dopamine by astrocytes in the developing prefrontal cortex leads to cognitive impairments
references.venue: Molecular Psychiatry,
references.authors: Tiina Manninen Ausra Saudargiene Marja-Leena Linne
references.title: Astrocyte-mediated spike-timingdependent long-term depression modulates synaptic properties in the developing cortex
references.venue: PLOS Computational Biology, 16(11):1–29,
references.authors: Jérémie Sibille Ulrike Pannasch Nathalie Rouach
references.title: Astroglial potassium clearance contributes to short-term plasticity of synaptically evoked currents at the tripartite synapse
references.venue: The Journal of Physiology,
references.authors: Rogier Min Thomas Nevian
references.title: Astrocyte signaling controls spike timing–dependent depression at neocortical synapses
references.venue: Nature Neuroscience,
references.authors: Alexander Stanley Thrane Vinita Rangroo Thrane Douglas Zeppenfeld Nanhong Lou Qiwu Xu Erlend Arnulf Nagelhus Maiken Nedergaard
references.title: General anesthesia selectively disrupts astrocyte calcium signaling in the awake mouse cortex
references.venue: Proceedings of the National Academy of Sciences,
references.authors: Jeannine Foley Tamara Blutstein SoYoung Lee Christophe Erneux Michael M. Halassa Philip Haydon
references.title: Astrocytic ip3/ca2+ signaling modulates theta rhythm and rem sleep
references.venue: Frontiers in Neural Circuits,
references.authors: Laura Bojarskaite Daniel M. Bjørnstad Klas H. Pettersen Céline Cunen Gudmund Horn Hermansen Knut Sindre Åbjørsbråten Rolf Sprengel Koen Vervaeke Wannan Tang Rune Enger Erlend A. Nagelhus
references.title: Astrocytic ca2+ signaling is reduced during sleep and is involved in the regulation of slow wave sleep
references.venue: Nature Communications,
references.authors: Ashley M. Ingiosi Christopher R. Hayworth Daniel O. Harvey Kristan G. Singletary Michael J. Rempe Jonathan P. Wisor Marcos G. Frank
references.title: A role for astroglial calcium in mammalian sleep and sleep regulation
references.venue: Current Biology,
references.authors: Enzo Tagliazucchi Dante R. Chialvo Michael Siniatchkin Enrico Amico Jean-Francois Brichant Vincent Bonhomme Quentin Noirhomme Helmut Laufs Steven Laureys
references.title: Largescale signatures of unconsciousness are consistent with a departure from critical dynamics
references.venue: Journal of The Royal Society Interface,
references.authors: Timothy Bellay Andreas Klaus Saurav Seshadri Dietmar Plenz
references.title: Irregular spiking of pyramidal neurons organizes as scale-invariant neuronal avalanches in the awake state. eLife
references.venue: ISSN 2050-084X. doi: 10.7554/eLife.07224
references.authors: Gerald Hahn Adrian Ponce-Alvarez Cyril Monier Giacomo Benvenuti Arvind Kumar Frédéric Chavane Gustavo Deco Yves Frégnac
references.title: Spontaneous cortical activity is transiently poised close to criticality
references.venue: PLOS Computational Biology, 13(5):1–29,
references.authors: Viola Priesemann Mario Valderrama Michael Wibral Michel Le Van Quyen
references.title: Neuronal avalanches differ from wakefulness to deep sleep – evidence from intracranial depth recordings in humans
references.venue: PLOS Computational Biology, 9(3):1–14,
references.authors: Erik D. Fagerholm Romy Lorenz Gregory Scott Martin Dinov Peter J. Hellyer Nazanin Mirzaei Clare Leeson David W. Carmichael David J. Sharp Woodrow L. Shew Robert Leech
references.title: Cascades and cognitive state: focused attention incurs subcritical dynamics
references.venue: Journal of Neuroscience,
references.authors: Vladimir Parpura Philip G. Haydon
references.title: Physiological astrocytic calcium levels stimulate glutamate release to modulate adjacent neurons
references.venue: Proceedings of the National Academy of Sciences,
references.authors: Catherine A. Christian John R. Huguenard
references.title: Astrocytes potentiate gabaergic transmission in the thalamic reticular nucleus via endozepine signaling
references.venue: Proceedings of the National Academy of Sciences,
references.authors: Sara Mederos Candela González-Arias Gertrudis Perea
references.title: Astrocyte–neuron networks: a multilane highway of signaling for homeostatic brain function
references.venue: Frontiers in Synaptic Neuroscience,
references.authors: Eiji Shigetomi Sandip Patel Baljit S. Khakh
references.title: Probing the complexities of astrocyte calcium signaling
references.venue: Trends in Cell Biology,
references.authors: Gertrudis Perea Mriganka Sur Alfonso Araque
references.title: Neuron-glia networks: integral gear of brain function
references.venue: Frontiers in Cellular Neuroscience,
references.authors: Y. Lecun L. Bottou Y. Bengio P. Haffner
references.title: Gradient-based learning applied to document recognition
references.venue: Proceedings of the IEEE,
references.authors: Garrick Orchard Ajinkya Jayawant Gregory K. Cohen Nitish Thakor
references.title: Converting static image datasets to spiking neuromorphic datasets using saccades
references.venue: Frontiers in Neuroscience,
references.authors: Han Xiao Kashif Rasul Roland Vollgraf
references.title: Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms, 2017
references.authors: Abigail Morrison Markus Diesmann Wulfram Gerstner
references.title: Phenomenological models of synaptic plasticity based on spike timing
references.venue: Biological Cybernetics,
references.authors: Wenrui Zhang Peng Li
references.title: Information-theoretic intrinsic plasticity for online unsupervised learning in spiking neural networks
references.venue: Frontiers in Neuroscience,
references.authors: Wolfgang Maass Robert Legenstein Nils Bertschinger
references.title: Methods for estimating the computational power and generalization capability of neural microcircuits
references.venue: In Proceedings of the 17th International Conference on Neural Information Processing Systems,
references.authors: Parami Wijesinghe Gopalakrishnan Srinivasan Priyadarshini Panda Kaushik Roy
references.title: Analysis of liquid ensembles for enhancing the performance and accuracy of liquid state machines
references.venue: Frontiers in Neuroscience,
references.authors: Joschka Boedecker Oliver Obst Joseph T Lizier N Michael Mayer Minoru Asada
references.title: Information processing in echo state networks at the edge of chaos
references.venue: Theory Biosci.,
references.authors: Andre S. Ribeiro Stuart A. Kauffman Jason Lloyd-Price Björn Samuelsson Joshua E.S. Socolar
references.title: Mutual information in random boolean models of regulatory networks
references.venue: Phys. Rev. E,
references.authors: Łukasz Kuśmierz Shun Ogawa Taro Toyoizumi
references.title: Edge of chaos and avalanches in neural networks with heavy-tailed synaptic weight distribution
references.venue: Phys. Rev. Lett.,
references.authors: C. van Vreeswijk H. Sompolinsky
references.title: Chaos in neuronal networks with balanced excitatory and inhibitory activity
references.venue: Science, 274(5293):1724–1726,
references.authors: Srdjan Ostojic
references.title: Two types of asynchronous activity in networks of excitatory and inhibitory spiking neurons
references.venue: Nature Neuroscience,
references.authors: Kanaka Rajan L.F. Abbott Haim Sompolinsky
references.title: Stimulus-dependent suppression of chaos in recurrent neural networks
references.venue: Phys. Rev. E,
references.authors: Bin Zhou Yun-Xia Zuo Ruo-Tian Jiang
references.title: Astrocyte morphology: diversity, plasticity, and role in neurological diseases
references.venue: CNS Neuroscience & Therapeutics,
references.authors: Dongcheng Zhao Yi Zeng Tielin Zhang Mengting Shi Fei fei Zhao
references.title: Glsnn: A multi-layer spiking neural network based on global feedback alignment and local stdp plasticity
references.venue: Frontiers in Computational Neuroscience,
references.authors: Arash Samadi T. Lillicrap D. Tweed
references.title: Deep learning with dynamic spiking neurons and fixed feedback weights
references.venue: Neural Computation,
references.authors: Tielin Zhang Yi Zeng Dongcheng Zhao Bo Xu
references.title: Brain-inspired balanced tuning for spiking neural networks
references.venue: In Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence,
references.authors: Peter Diehl Matthew Cook
references.title: Unsupervised learning of digit recognition using spike-timingdependent plasticity
references.venue: Frontiers in Computational Neuroscience,
references.authors: Hesham Mostafa
references.title: Supervised learning based on temporal coding in spiking neural networks
references.venue: IEEE Transactions on Neural Networks and Learning Systems,
references.authors: Maryam Mirsadeghi Majid Shalchian Saeed Reza Kheradpisheh Timothée Masquelier
references.title: Stidi-bp: Spike time displacement based error backpropagation in multilayer spiking neural
references.venue: networks. Neurocomputing,
references.authors: Jun Haeng Lee Tobi Delbruck Michael Pfeiffer
references.title: Training deep spiking neural networks using backpropagation
references.venue: Frontiers in Neuroscience,
references.authors: Yujie Wu Lei Deng Guoqi Li Jun Zhu Luping Shi
references.title: Spatio-temporal backpropagation for training high-performance spiking neural networks
references.venue: Frontiers in Neuroscience,
references.authors: Jacques Kaiser Hesham Mostafa Emre Neftci
references.title: Synaptic plasticity dynamics for deep continuous local learning (decolle)
references.venue: Frontiers in Neuroscience,
references.authors: Qianhui Liu Haibo Ruan Dong Xing H. Tang Gang Pan
references.title: Effective aer object classification using segmented probability-maximization learning in spiking neural networks
references.venue: In AAAI,
references.authors: S. Shrestha G. Orchard
references.title: Slayer: Spike layer error reassignment in time
references.venue: In NeurIPS,
references.authors: Tielin Zhang Yi Zeng Dongcheng Zhao Mengting Shi
references.title: A plasticity-centric approach to train the non-differential spiking neural networks
references.venue: In Proceedings of the AAAI Conference on Artificial Intelligence,
references.authors: Yunzhe Hao Xuhui Huang Meng Dong Bo Xu
references.title: A biologically plausible supervised learning method for spiking neural networks using the symmetric stdp rule
references.venue: Neural Networks,
references.authors: Saeed Reza Kheradpisheh Maryam Mirsadeghi Timothée Masquelier
references.title: BS4NN: binarized spiking neural networks with temporal coding and learning
references.authors: Han Ju Jian-Xin Xu Edmund Chong Antonius M.J. VanDongen
references.title: Effects of synaptic connectivity on liquid state machine performance
references.venue: Neural Networks,
references.authors: Filip Ponulak Andrzej Kasiński
references.title: Supervised learning in spiking neural networks with resume: Sequence learning, classification, and spike shifting
references.venue: Neural Comput.,
references.authors: Robert Urbanczik Walter Senn
references.title: A Gradient Learning Rule for the Tempotron
references.venue: Neural Computation, 21(2):340–352,
references.authors: Robert Gütig Haim Sompolinsky
references.title: The tempotron: a neuron that learns spike timing–based decisions
references.venue: Nature Neuroscience,
references.authors: Guangzhi Tang Ioannis E. Polykretis Vladimir A. Ivanov Arpit Shah Konstantinos P. Michmizos
references.title: Introducing astrocytes on a neuromorphic processor: Synchronization, local plasticity and edge of chaos
references.venue: ACM Proceedings of 2019 Neuroinspired Computing Elements (NICE 2019),
references.authors: Alex Roxin Nicolas Brunel David Hansel Gianluigi Mongillo Carl van Vreeswijk
references.title: On the distribution of firing rates in networks of cortical neurons
references.venue: Journal of Neuroscience,
references.authors: Woodrow L. Shew Hongdian Yang Thomas Petermann Rajarshi Roy Dietmar Plenz
references.title: Neuronal avalanches imply maximum dynamic range in cortical networks at criticality
references.venue: Journal of Neuroscience,
references.authors: Friedemann Zenke Guillaume Hennequin Wulfram Gerstner
references.title: Synaptic plasticity in neural networks needs homeostasis with a fast rate detector
references.venue: PLOS Computational Biology, 9(11):1–14,
references.authors: Michael M. Halassa Philip G. Haydon
references.title: Integrated brain circuits: astrocytic networks modulate neuronal activity and behavior
references.venue: Annual Review of Physiology,
references.authors: Eduardo E. Benarroch
references.title: Neuron-astrocyte interactions: partnership for normal function and disease in the central nervous system
references.venue: Mayo Clinic Proceedings,
references.authors: Dionysia T. Theodosis Dominique A. Poulain Stéphane H.R. Oliet
references.title: Activity-dependent structural and functional plasticity of astrocyte-neuron interactions
references.venue: Physiological Reviews,
sections.heading: 1 Introduction
sections.text: With the recent rise of neuromorphic [1–4] and edge computing [5, 6], the liquid state machine (LSM) learning framework [7] has become an attractive alternative [8–11] to deep neural networks owing to its compatibility with energy-efficient neuromorphic hardware [12–14] and inherently low training complexity. Originally proposed as a biologically plausible model of learning, LSMs avoid training via backpropagation by using a sparse, recurrent, spiking neural network (liquid) with fixed synaptic connection weights to project inputs into a high dimensional space from which a single neural layer can learn the correct outputs. Yet, these advantages over deep networks come at the expense of 1) sub-par accuracy and 2) extensive data-specific hand-tuning of liquid weights. Interestingly, these
  1Code and data available at https://github.com/combra-lab/NALSM
  35th Conference on Neural Information Processing Systems (NeurIPS 2021).
  two limitations have been targeted by several studies that tackle one [15, 16] or the other [17, 18], but not both. This has limited the widespread use of LSMs in real-world applications [8]. In that sense, there is an unmet need for a unified, brain-inspired approach that is directly applicable to the emerging neuromorphic and edge computing technologies, facilitating them to go mainstream.
  As a general heuristic, LSM accuracy is maximized when LSM dynamics are positioned at the edgeof-chaos [19–21] and specifically in the vicinity of a critical phase transition [22–25] that separates: 1) the sub-critical phase, where network activity decays, and 2) the super-critical (chaotic) phase, where network activity gets exponentially amplified. Strikingly, brain networks have also been found to operate near a critical phase transition [26–28] that is modeled as a branching process [25, 26]. Current LSM tuning methods organize network dynamics at the critical branching factor by adding forward and backward communication channels on top of the liquid [15, 16]. This, however, results in significant increases in training complexity and violates the LSM’s brain-inspired self-organization principles. For example, these methods lack local plasticity rules that are widely observed in the brain and considered a key component for both biological [29] and neuromorphic learning [3, 2, 4]. A particular local learning rule, spike-timing-dependent plasticity (STDP), is known to improve LSM accuracy [17, 18]. Yet, current methods of incorporating STDP into LSMs further exacerbate the limitations of data-specific hand-tuning as they require additional mechanisms to compensate for the STDP-imposed saturation of synaptic weights [17, 30–33]. This signifies the scarcity of LSM tuning methods that are both computationally efficient and data-independent.
  A long-neglected non-neuronal cell in the brain, astrocytes, is now known to play key roles in modulating brain networks [34–39], from modifying synaptic plasticity [40–42] to facilitating switching between cognitive states [43–46] that have been linked to a narrow spectrum of dynamics around the critical phase transition [47–51]. The mechanisms that astrocytes use to modulate neurons include the integration of the activity of thousands of synapses into a slow intracellular continuous signal that feeds back to neurons by affecting their synaptic plasticity [52–55, 42]. The unique spatiotemporal attributes [56, 57] identified in astrocytes align well with the brain’s remarkable ability to self-organize its massive and highly recursive networks near criticality. That is why astrocytes present a fascinating possibility of forming a unified feedback modulation mechanism required to improve baseline LSM accuracy while eliminating data-specific hand-tuning.
  Here, we propose the neuron-astrocyte liquid state machine (NALSM), where a biologically inspired astrocyte model organized liquid dynamics near a critical phase transition, by modulating STDP. We show that NALSM combined the computational benefits of both STDP and critical branching dynamics by demonstrating its accuracy advantage compared to other LSM methods on two datasets: 1) MNIST [58], and 2) N-MNIST [59]. We demonstrate that, similar to its biological counterpart that handles new and unstructured information with robustness and versatility, NALSM maintains the state-of-the-art LSM performance without re-tuning training parameters for each tested dataset. We also show that a NALSM with a large enough liquid can attain comparable accuracy to fully-connected multi-layer spiking neural networks trained via backpropagation on 1) MNIST [58], 2) N-MNIST [59], as well as 3) Fashion-MNIST [60]. Our results suggest that the under-performance and high training difficulty of current neuromorphic methods can be addressed by harvesting neuroscience knowledge and further translating biological principles to computational mechanisms.
sections.heading: 2 Methods
sections.heading: 2.1 The neuron-astrocyte liquid state machine
sections.text: To construct the NALSM, we started with a baseline LSM model consisting of 2 layers: 1) a spiking liquid, and 2) a linear output layer. Next, we added STDP to the LSM liquid, forming the LSM+STDP model. We developed a biologically faithful leaky-integrate-and-modulate (LIM) astrocyte model, which we embedded in the LSM+STDP liquid, to form the NALSM. The process is formalized below.
  LSM Model We implemented the baseline LSM as a 3-dimensional neural network (liquid) consisting of 1, 000 neurons surrounded by 1-dimensional layers of input and output neurons. Number of input neurons was 784 and 2, 312 for MNIST and N-MNIST, respectively (See Appendix A.1). We used the leaky-integrate-and-fire (LIF) model [3] for input and liquid neurons, modeled as:
  dvi dt = − 1 τv vi(t) + ui(t)− θiσi(t) (1)
  ui(t) = ∑ j 6=i wij(αu ∗ σj)(t) + bi (2)
  where vi is the membrane potential and ui is the synaptic response current of neuron i, θi is the membrane potential threshold, σi(t) = ∑ k δ(t − tki ) is the spike train of neuron i with tki being the time of the k-th spike, wij is the weight connecting neuron j to i, bi is the bias of neuron i, and αu(t) = τ−1u exp(−t/τu)H(t) is the synaptic filter with H(t) being the unit step function (See Appendix A.2). All LIF neurons had a 2 ms absolute refractory period. Liquid neurons were excitatory and inhibitory with 80%/20% ratio. Input neurons did not have an excitatory/inhibitory distinction and had random excitatory and inhibitory connections to liquid neurons. From here on, we will refer to connections between input neurons and liquid neurons as IL connections, inter-liquid connections as LL, and liquid to output connections as LO. In line with [7], we created LL connections using probabilities based on Euclidean distance, D(i, j), between any two neurons i, j:
  P (i, j) = C · exp ( − ( D(i, j)
  λ
  )2) (3)
  with closer neurons having higher connection probability. Parameters C and λ set the amplitude and horizontal shift, respectively, of the probability distribution (See Appendix A.3). Density of IL connections was 15%. The output layer was a dense layer consisting of 10 linear neurons.
  LSM+STDP Model We added unsupervised, local learning to the LSM model by letting STDP change each LL and IL connection [61], modeled as:
  dw dt = A+Tpre ∑ o δ(t− topost)−A−Tpost ∑ i δ(t− tipre) (4)
  where A+ = A− = 0.15 are the potentiation/depression learning rates and Tpre/Tpost are the pre/post-synaptic trace variables, modeled as,
  τ∗+ dTpre dt = −Tpre + a+ ∑ i δ(t− tipre) (5)
  τ∗− dTpost dt = −Tpost + a− ∑ o δ(t− topost) (6)
  where a+ = a− = 0.1 are the discrete contributions of each spike to the trace variable, τ∗+ = τ ∗ − = 10 ms are the decay time constants, tipre and t o post are the times of the pre-synaptic and post-synaptic spikes, respectively. We constrained connection weights to: 1) IL: [−3, 3] , 2) excitatory LL: [0, 3], and 3) inhibitory LL: [−3, 0]. We used the same STDP parameters for all models and experiments.
  LIM Astrocyte Model We developed the astrocyte model as a leaky integrator with a continuous output value Aastro− , expressed as:
  τasto dAastro− dt = −Aastro− + wastro ∑ i∈Nliq δ(t− ti)− wastro ∑ j∈Ninp δ(t− tj) + bastro (7)
  where Aastro− directly mapped to A− in equation (4), bastro = A+ adjusted the astrocyte output to the fixed STDP potentiation learning rate, Nliq and Ninp are the sets of liquid and input neurons, respectively, andwastro set astrocyte responsiveness to network activity (See Appendix A.4). Ignoring the decay and bias terms, the astrocyte model computed the difference in the number of spikes produced by liquid neurons and input neurons. Functionally, this is equivalent to computing the ratio of spikes emitted by the liquid over the input neurons:
  BFproxy(t) = ∑ i∈Nliq δ(t− ti)∑ j∈Ninp δ(t− tj)
  (8)
  Specifically, when the liquid produced more spikes than the input neurons, their difference was positive which translated to BFproxy > 1.0, and vice versa. This approach to measure liquid dynamics acted as a network level approximation of the branching factor, σBF , which is normally
  evaluated for each neuron (See 2.3.1). We empirically confirmed that BFproxy = 1.0 aligned with the critical branching factor, σBF = 1.0 (Fig. 1 B). Hence, as dynamics became progressively supercritical (σBF > 1.0), BFproxy became greater than 1, which caused the LIM astrocyte to increase STDP depression learning rate above the fixed STDP potentiation learning rate (Aastro− → A− > A+). This caused STDP to decrease the average weight of LL and IL connections, which decreased number of spikes produced by the liquid and made dynamics less super-critical (Fig. 1 B). The reverse occurred as dynamics became progressively sub-critical. As a result of astrocyte modulation, liquid dynamics oscillated between sub-critical and super-critical until eventual stabilization near the critical branching factor (See Appendix A.4).
  NALSM Model We completed NALSM by adding the LIM astrocyte to the LSM+STDP model’s liquid (Fig. 1 A). As described above, the LIM astrocyte integrated activity from input and liquid neurons, and continuously controlled the STDP depression learning rate.
sections.heading: 2.2 Training
sections.text: Model training was done in 3 steps: 1) initialization of IL and LL liquid connections, 2) passing all data through the liquid resulting in liquid neuron spike counts, and 3) training the output layer on the spike counts. The steps are further detailed below.
sections.heading: 2.2.1 Liquid initialization
sections.text: LSM We initialized all IL and LL connections with a single weight value, maintaining originally defined connection signs [62]. Weights were constant during spike count collection.
  LSM+STDP We initialized IL and LL connections with STDP by consecutively presenting all the training images to the liquid. Starting with initially maximal connections, 3(−3) for excitatory(inhibitory) connections, we let STDP continuously adjusted weights while presenting the liquid with a randomly ordered series of MNIST training image snapshots, each lasting 20 ms. For N-MNIST, we randomly sampled each 20 ms snapshot from the 0 − 250 ms range, as a way to account for the variability in the temporal dimension (See 2.3). In each case, we used a total of 50, 000 snapshots, each corresponding to a unique training image. We used STDP only for weight initialization. Initialized weights were fixed during spike count collection.
  NALSM We used the LSM+STDP weight initialization process with the exception of added STDP modulation by the LIM astrocyte. We used this set of initialized weights as the starting point for each sample in the spike counting phase, during which astrocyte-modulated STDP continued to adjust synaptic weights to compensate for slight deviations in dynamics caused by each input sample’s different level of activity. For each sample, parameters A+ from (4) and bastro from (7) were both initialized to 0.15 and decayed at a rate of 0.99 for the duration of sample input.
sections.heading: 2.2.2 Output layer training
sections.text: We assembled spike counts by presenting each sample image to the liquid for 250 ms and counting the number of spikes emitted by each liquid neuron for the full duration of input. We used Adam optimizer to batch train the output layer on spike count vectors by minimizing the cross entropy loss with L2-regularization,
  L(yi, ŷ) = − 1
  m m∑ i=1 yilog(ŷi) + (1− yi)log(1− ŷi) + λreg 2m ||Wout||2F (9)
  where m = 250 is the batch size, Wout is the output layer weight matrix, λreg = 5× 10−10 is the regularization hyperparameter, yi and ŷi are the normalized vectors denoting the predicted label and the target label, respectively. Prior to training, we initialized output layer weights/biases to 0.0, and the learning rate to 0.1. We trained the output layer until validation accuracy peaked (up to a maximum of 5, 000 epochs), at which point we evaluated model test accuracy.
sections.heading: 2.3 Experiments
sections.text: We performed all LSM comparison experiments on MNIST and N-MNIST datasets (See A.1). Using 10 randomly generated networks for each dataset, we trained 1) the baseline LSM model, 2) the LSM+STDP model, 3) the NALSM model (See 2.1), and 4) the LSM+AP-STDP model, a method for incorporating STDP in the LSM liquid [17] (See Appendix A.5). First, we evaluated LSM model accuracy with respect to liquid weight, which ranged in 0.4 − 1.2 for MNIST and 0.8 − 1.35 for N-MNIST. We used a random seed for each training session. Next, we evaluated corresponding network dynamics of each network/weight combination by measuring the liquid’s branching factor on 20 randomly sampled inputs (See 2.3.1). To have comparable results for each network, we trained the remaining models using the same seed that resulted in peak LSM accuracy. For NALSM, we used the same initialization and parameters for all networks and datasets. For LSM+AP-STDP, we hand-tuned STDP control parameters for each network and dataset combination to maximize validation accuracy (See Appendix A.5). Additionally, we trained NALSM on Fashion-MNIST dataset (See A.1) using the same 10 randomly generated networks that we had used for MNIST.
  Sparse neuron-astrocyte connectivity We tested NALSM’s accuracy as a function of neuronastrocyte connection density on 3 best performing networks (per dataset). Keeping the proportion of neurons sampled by the astrocyte the same for both input neurons and liquid neurons, we trained NALSM with 10%, 20%, 40%, 60%, and 80% neuron-astrocyte density over 3 seeds for each of 3 networks. Regardless of connection sparseness, all IL/LL connections were modulated by Aastro− (See 2.1)
  NALSM with larger liquid sizes We tested NALSM performance for larger liquids. For each size, we trained 3 randomly generated networks, each on a random seed. All parameters and initialization were same as for 1, 000 neuron liquid. For maximum accuracy, we trained NALSM with an 8, 000 neuron liquid. For each dataset, we used 5 randomly generated networks trained on a random seed. Parameter wastro = 0.0075 for all datasets. All other parameters and initialization were as before.
sections.heading: 2.3.1 Branching factor of liquid
sections.text: To evaluate a liquid’s dynamics, we used the network branching factor, σBF , which quantifies network information flow amplification/decay. Liquid dynamics are sub-critical, near-critical and super-critical when σBF < 1.0, σBF ≈ 1.0, and σBF > 1.0. We calculated σBF as done in [30], with offset φ = 0 and time window ∆ = 4 ms as per [26].
sections.heading: 2.3.2 Kernel quality of liquid
sections.text: We evaluated liquid 1) linear separation, and 2) generalization capability using methods from [63]. For the MNIST and N-MNIST test sets, we computed the rank of matrix M assembled from k randomly selected spike count vectors, resulting in shape Nliq × k. We repeated this on 1, 000 shuffles of spike vectors. For linear separation, we used spike counts from model testing phase. For generalization capability, we added noise to input data and evaluated new spike count vectors (See 2.2.2). For MNIST, we added N (0, 125) noise to each pixel value. For N-MNIST, we time-shifted each event by N (0, 10). Taken together across all models and both datasets, we rescaled ranks of each measure to 0− 1 range and subtracted measure 1 from measure 2 as in [63]. Due to negative differences, we again rescaled all differences to 0− 1.
sections.heading: 3 Results
sections.heading: 3.1 Baseline LSM performance
sections.text: We established a benchmark accuracy for the baseline LSM on MNIST and N-MNIST datasets (See 2.3). We acquired our baseline by averaging over 10 randomly generated liquids with 1, 000 neurons (See 2.1). The LSM achieved a top accuracy of 95.44% (95.30± 0.11%) on MNIST, and 95.35% (95.02± 0.15%) on N-MNIST (See 2.2). For MNIST, this was comparable to the previously reported state-of-the-art LSM accuracy [64], using the same sized liquid. Further, LSM accuracy was very sensitive to the liquid’s weight (Fig. 2 A).
sections.heading: 3.2 LSM performance peaked at the critical branching factor
sections.text: The peak LSM accuracy on each dataset corresponded to a different liquid synaptic weight. Specifically, there were cases where a liquid with weights tuned for maximum accuracy on MNIST, would catastrophically fail on N-MNIST (Fig. 2 A). Also, LSM accuracy on MNIST plateaued for a wider
  range of weights than on N-MNIST, which can be attributed to N-MNIST’s greater difficulty caused by its variability over the temporal dimension (See Appendix A.1). Taken together, this indicated that LSM training requires extensive hand-tuning of weights for each specific dataset.
  Since critical dynamics are well known to result in near-maximum LSM performance [65, 19–21], dataset-specific hand-tuning can be significantly reduced by replacing accuracy with liquid dynamics as the target output of weight tuning. Indeed, LSM accuracy was near-maximum for both datasets, when the liquid’s branching factor was in 1.0− 1.2 range, or slightly super-critical (Fig. 2 B) (See 2.3.1). This agrees with studies showing that information transfer in finite sized systems peaks at slightly super-critical dynamics [66]. Although each dataset still had different weight ranges corresponding to the critical branching factor, the relationship between liquid dynamics and weight was positive for both datasets (Fig. 2 C). Known to generalize beyond specific datasets [20], this relationship suggested that near-critical dynamics can be organized using STDP, by providing it directional feedback from current liquid dynamics.
sections.heading: 3.3 Astrocyte-modulated plasticity organized liquid dynamics near criticality
sections.text: The LIM astrocyte model stabilized liquid dynamics near the critical branching factor as we presented a continuous stream of samples to the neuron-astrocyte liquid (Fig. 2 B). The NALSM’s slightly super-critical stabilization suggested that liquid dynamics were at the edge-of-chaos. While chaotic activity is known to correspond to super-critical branching dynamics in some cases [25], such correspondence is not guaranteed. Hence, we examined additional network properties that are necessary and indicative of chaotic activity (See Appendix A.6). Specifically, the astrocyte-modulated liquid had coexistence of small and large synaptic weights (Fig. S2), as well as a balance of excitation and inhibition, both of which are necessary for the existence of chaotic network activity [67, 68]. Further supporting a chaotic activity, the neuron-astrocyte liquid spike activity appeared irregular (Fig. S3). We also performed autocorrelation analysis on liquid neuron spike trains, which further suggested the existence of chaotic activity with a correspondence between edge-of-chaos dynamics and critical branching dynamics (Fig. S4) [69, 70] (See Appendix A.6). Given that liquid dynamics were directly approximated by the LIM astrocyte, which directly controlled the STDP depression learning rate (See 2.1), NALSM required no dataset-specific hand-tuning. As a result, we used the same weight initialization and parameters to benchmark NALSM (See 2.3).
sections.heading: 3.4 Benchmarking NALSM performance on MNIST and N-MNIST
sections.text: On both datasets, NALSM achieved superior performance to comparable LSM models of the same size. Using 1, 000 liquid neurons, NALSM achieved a top accuracy of 96.15% (95.96 ± 0.13%) on MNIST and 96.13% (95.90± 0.16%) on N-MNIST; outperforming LSM model’s top accuracy by 0.71% on MNIST and 0.78% on N-MNIST (Fig. 3 A). We also compared NALSM to a stateof-the-art LSM STDP method, AP-STDP (See 2.3). The LSM+AP-STDP model required more extensive dataset-specific hand-tuning than the baseline LSM due to its additional STDP control parameters (See Appendix A.5). Resulting in top accuracy of 95.62% (95.49± 0.09%) and 95.43% (95.23± 0.16%), the LSM+AP-STDP model was superseded by NALSM by 0.53% and 0.70% on MNIST and N-MNIST, respectively. As a control measure, we also trained a LSM with unregulated STDP (See 2.1). The LSM+STDP model significantly under-performed compared to all other models achieving a top accuracy of 90.52% (89.47± 0.45%) on MNIST and 87.71% (86.80± 0.51%) on N-MNIST. We attributed this under-performance to the LSM+STDP liquid’s excessive super-critical dynamics (Fig. 2 B) which are well known to decrease liquid computational capacity [65, 66].
  The NALSM had the most robust accuracy performance across the two datasets out of all the compared LSM models. With no dataset-specific tuning, NALSM’s average accuracy on N-MNIST was lower than the accuracy on MNIST by only −0.05%. This was 5− 50 times less than for the LSM+AP-STDP (−0.26%), LSM (−0.29%), and LSM+STDP (−2.66%) models. We attributed the NALSM’s performance advantage to the improved computational properties of its liquid. For both tested datasets, the NALSM achieved slightly super-critical branching dynamics where baseline LSM performance peaked right before it started to decline with increasing supercritical dynamics (Fig. 2 B). This suggested that NALSM’s performance advantage, compared to a LSM with similar dynamics, was due to the addition of astrocyte-modulated STDP (See 2.1, 2.2.1). While LSM+AP-STDP and LSM+STDP models also had STDP, their lower performance can be explained by their excessively sub-critical and super-critical dynamics, respectively (Fig. 2 B). We further confirmed that NALSM’s increased performance resulted from the improved computational properties of its liquid by measuring each model’s liquid kernel quality. This encompassed both the linear separation and generalization capability of the liquid (See 2.3.2). Higher model accuracy corresponded to higher kernel quality for all 4 models (Fig. 3 B). This is a further indication that near-critical dynamics and astrocyte-modulated STDP contributed to the NALSM’s performance increase.
sections.heading: 3.5 NALSM maintained performance with sparse neuron-astrocyte connectivity
sections.text: The NALSM maintained its accuracy advantage even with neuron-astrocyte connection densities as low as 10% (Fig. 4). In the brain, astrocytes contact only approximately 65% of all synapses in their surroundings [71]. We tested NALSM performance as a function of neuron-astrocyte connection density (See 2.3). The NALSM mean accuracy decreased marginally with increasingly sparse connectivity, while variability in performance was minimal across densities. At 10% connectivity, average NALSM accuracy decreased by 0.36% for MNIST and 0.14% for N-MNIST compared to 100% connection density. In both cases, average NALSM accuracy was still above LSM and LSM+APSTDP average accuracy.
sections.heading: 3.6 Larger liquids increased NALSM accuracy
sections.text: The NALSM accuracy improved with increased liquid size, saturating at approximately 8, 000 neurons (See Appendix A.7). NALSM8000 achieved a top accuracy of 97.61% (97.49± 0.11%) on MNIST, 97.51% (97.42± 0.07%) on N-MNIST, and 85.84% (85.61± 0.18%) on Fashion-MNIST. Compared to previously reported benchmarks on MNIST and Fashion-MNIST, the NALSM8000 outperformed all brain-inspired learning methods that do not use backpropagation of gradients or its approximation through feedback alignment [72, 73], with the exception of [74] for MNIST. While [74] demonstrated that a fully-connected 2-layer spiking network can achieve high accuracy through a combination of biologically-plausible plasticity rules, it is not clear how such an approach would scale to more layers without some form of backpropagation. Conversely, multi-layered LSMs have been shown to work without backpropagation [8, 10]. Further, NALSM8000 used approximately 1/3 (≈ 1, 199, 407 ± 453) of number of trainable(plastic) connections as in [74]. Compared to top accuracies reported for fully-connected multi-layered spiking neural networks trained with backpropagation, the NALSM8000 achieved comparable performance on all datasets; outperforming multiple reported results on MNIST and N-MNIST (Table 1) (See Appendix A.8).
sections.heading: 4 Discussion and Broader Impact
sections.text: Ironically, LSMs are one of the most brain-like and at the same time one of the most difficult to train learning models. Here, we proposed an astrocyte model that merged critical branching dynamics and STDP into a single liquid, thereby simultaneously improving LSM performance and decreasing data-specific tuning. We showed that the synergy of STDP and near-critical branching dynamics
  improved the computational capacity of the liquid, which translated to better than state-of-the-art LSM accuracy on MNIST and N-MNIST, and do so with minimal added computational cost (See Appendix A.9). Our results indicate that, given a large enough liquid, NALSM performance compares to current fully-connected multi-layer spiking neural networks trained via backpropagation.
  The reported narrowing of the performance gap between brain-inspired LSM and deep networks suggests that studying the interaction among the brain’s computational principles can help our learning models to reach human-like performance. Indeed, our results demonstrate that the synergy of braininspired astrocyte-modulated STDP and near-critical dynamics resulted in the superior performance of NALSM compared to 1) a LSM with critical dynamics but without STDP, and 2) a LSM with STDP, but without critical dynamics. Aligning with other studies showing that liquid topology impacts LSM accuracy [87], we also showed that a brain-inspired, sparse, 3D-distance-based network architecture can improve the computational capacity of a single liquid. Specifically, our baseline 3D LSM achieved comparable accuracy to the multi-liquid LSM [64], which improved performance of a single dimensionless liquid by partitioning it into multiple liquids. While we demonstrated NALSM performance using only the 3D-distance-based network architecture, our proposed astrocyte modulation method does not depend on network topology and, therefore, is applicable to other types of topology. In fact, our approach is also extendable to multi-liquid architectures and other local plasticity rules that follow STDP’s separation of potentiation and depression components.
  The astrocyte-modulated LSM learning framework is also compatible with the emerging neuromorphic hardware. This is because the gradient descent that we used for training the linear output can be replaced by a single-layer spike based learning rule [88–90]. This makes NALSM compatible with neuromorphic hardware, exploiting in full its advantages [91]. For example, our method can leverage even further the energy efficiency of neuromorphic chips, by virtue of its low spiking rates. In line with biological ranges [92], NALSM had spiking rates that ranged from 12 Hz to 37 Hz, depending on the input sample. These rates can be reduced further, by modifying input encoding, since liquid spiking rates are directly adjusted by the astrocyte based on input spiking rates (Fig. 1).
  Here, we demonstrated a possible connection between the near-critical branching dynamics of the NALSM liquid and the edge-of-chaos transition (See Appendix A.6). The critical branching transition has been extensively used to model critical dynamics in brain networks [27, 26]. Focusing on the computational benefits of criticality, machine learning has mostly examined network dynamics at the edge-of-chaos transition. Although the presence of one transition does not guarantee the existence of the other, both transitions are well connected to the same result, an improved computational performance [25]. Indeed, the computational performance of systems poised at a critical phase transition has been widely studied both experimentally [22] and theoretically [23], and are wellconnected to both edge-of-chaos [19, 20] and critical branching transitions [25, 15]. Networks operating at near-criticality are believed to have simultaneous access to the computational properties (learning and memory) of both phases, which results in 1) maximizing their information processing capacity [22], 2) optimizing their dynamical range [93, 24], and 3) expanding their number of metastable states [25]. Hence, it is not surprising that the NALSM’s astrocyte imposed near-critical branching dynamics resulted in improved accuracy and generalization capabilities as observed in LSMs with edge-of-chaos dynamics [19, 20], while adding the benefit of a neuromorphic compatibility and self-organized criticality.
  Our work shows how insights from modern cellular neuroscience can synergize with neuromorphic computing, and lead to novel intelligent systems, spurring the dialogue between artificial intelligence and brain sciences. Indeed, given that the known neuronal mechanisms are too slow and uncoordinated in the brain to modulate STDP [31, 94, 32], it is an open question how neurons modulate synaptic plasticity. Our demonstration that the distinct temporal and spatial mechanisms of astrocytes may modulate STDP and subsequently regulate network dynamics, questions the neuron as the only processing unit in the brain [95–97]. In that sense, it helps in dismantling the 100-year old dogma that “brain = neurons”, and tackle the absence of astrocytes in both prevailing computational hypotheses on how the brain learns and efforts to translate such knowledge to effective models of intelligence.
  By showing how astrocyte-modulated STDP can maximize computational performance near criticality, we aimed to broaden the applicability of the LSM to complex spatio-temporal problems that require integration of data over multiple sources and time-scales, thereby, making LSMs suitable for real-life applications of edge computing. Our so far results suggest that this is a direction worth pursuing.
sections.heading: Acknowledgements
sections.text: This work is supported by the National Center for Medical Rehabilitation Research (NIH/NICHD) K12HD093427 Grant and by the Rutgers Office of Research and Innovation. Any findings, conclusions, and opinions expressed in this material are those of the authors and do not necessarily reflect the views of the NIH or Rutgers University.
title: Increasing Liquid State Machine Performance with Edge-of-Chaos Dynamics Organized by Astrocyte-modulated Plasticity