First, we compute the entropy of the probability function of hidden neurons in the controller. Due to computational constraints (the number of states of the probability distribution increases with \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${2}^{{N}_{h}}$$\end{document}2Nh) we only compute the entropy in agents up to Nh = 16 hidden units. Displaying the entropy of the neurons for different values of β we observe that the agents are near an order-disorder transition (Fig. 4A,B). Larger sizes make the transition sharper and closer to the operating temperature β = 1. From the entropy of a variable, its heat capacity can be computed as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$C(\beta )=-\,\beta \frac{{\rm{\partial }}H(\beta )}{{\rm{\partial }}\beta }$$\end{document}C(β)=−β∂H(β)∂β. From the computed 101 values of entropy, H(β) is estimated by fitting a curve using cubic B-splines25 as indicated in the Methods section. In Fig. 4C,D we observe how the system displays a similar divergence of the heat capacity of the neural network as the Ising model, suggesting that the robot’s neural controller is at a critical point.Figure 4(A,B) Entropy of the agent’s neural controller for different sizes up to N = 22 (Nh = 16), for 10 different agents and 101 values of β. (C,D) Heat capacity of the agent’s neurons. Heat capacity computed as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$C(\beta )=-\,\beta \frac{\partial H(\beta )}{\partial \beta }$$\end{document}C(β)=−β∂H(β)∂β, where H(β) is estimated using B-splines. Maximum and minimum values are shown by the grey area. The figures suggest that in both the Mountain Car (left) and Acrobot (right) embodiments the model presents a divergence of its heat capacity as the number of neurons N increases, suggesting that the neural controller of the system is near a continuous phase transition.