  We provide a refined characterization of the super-Turing computational power
of analog, evolving, and stochastic neural networks based on the Kolmogorov
complexity of their real weights, evolving weights, and real probabilities,
respectively. First, we retrieve an infinite hierarchy of classes of analog
networks defined in terms of the Kolmogorov complexity of their underlying real
weights. This hierarchy is located between the complexity classes $\mathbf{P}$
and $\mathbf{P/poly}$. Then, we generalize this result to the case of evolving
networks. A similar hierarchy of Kolomogorov-based complexity classes of
evolving networks is obtained. This hierarchy also lies between $\mathbf{P}$
and $\mathbf{P/poly}$. Finally, we extend these results to the case of
stochastic networks employing real probabilities as source of randomness. An
infinite hierarchy of stochastic networks based on the Kolmogorov complexity of
their probabilities is therefore achieved. In this case, the hierarchy bridges
the gap between $\mathbf{BPP}$ and $\mathbf{BPP/log^*}$. Beyond proving the
existence and providing examples of such hierarchies, we describe a generic way
of constructing them based on classes of functions of increasing complexity.
For the sake of clarity, this study is formulated within the framework of echo
state networks. Overall, this paper intends to fill the missing results and
provide a unified view about the refined capabilities of analog, evolving and
stochastic neural networks.

Computational Complexity;Artificial Intelligence