(A,B) Entropy of the agent’s neural controller for different sizes up to N = 22 (Nh = 16), for 10 different agents and 101 values of β. (C,D) Heat capacity of the agent’s neurons. Heat capacity computed as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$C(\beta )=-\,\beta \frac{\partial H(\beta )}{\partial \beta }$$\end{document}C(β)=−β∂H(β)∂β, where H(β) is estimated using B-splines. Maximum and minimum values are shown by the grey area. The figures suggest that in both the Mountain Car (left) and Acrobot (right) embodiments the model presents a divergence of its heat capacity as the number of neurons N increases, suggesting that the neural controller of the system is near a continuous phase transition.