In order to confirm a divergence in the observed transition, larger systems should be evaluated. Since evaluating the entropy of the hidden neurons is computationally infeasible for large sizes, we repeat the analysis for the state of the 4 sensor units of the network for a larger amount of hidden units. In Fig. 5 we show the entropy and heat capacity of the sensor units for sizes up to Nh = 64 hidden neurons, where we can observe a similar picture than in Fig. 4, suggesting that the heat capacity diverges, and a second order transition takes place in the neural controller of the agent. These results suggest that the agent’s neural controller is operating near a point of criticality, resembling a continuous phase transition, suggesting that the neural controller self-organizes to present maximal sensitivity to changes in its parameter space.Figure 5(A,B) Entropy of the sensor units for different sizes up to N = 70 (Nh = 64), for 10 different agents and 101 values of β. (C,D) Heat capacity of the agent’s neurons. Heat capacity computed as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$C(\beta )=-\,\beta \frac{{\rm{\partial }}H(\beta )}{{\rm{\partial }}\beta }$$\end{document}C(β)=−β∂H(β)∂β, where H(β) is estimated using B-splines. Maximum and minimum values are shown by the grey area. The figures suggest that in both the Mountain Car (left) and Acrobot (right) embodiments the system presents a divergence of its heat capacity as the number of neurons N increases, suggesting that the neural controller of the system is near a second order phase transition.