In order to evaluate the behaviour of the proposed learning rule, we test it in two embodied situations using the OpenAI Gym toolkit22. We define a neural network consisting of an Ising model defined as in Equation 2, describing a network of N = 6 + Hh units, with Nh hidden units, 2 motor units and 4 sensor units. Motor units define the actions performed by the agents. In sensor units, the magnetic field of the unit is not a fixed parameter but it is be updated with the value of an external input hi = Ii. Sensor units and motor units are only connected to hidden neurons, while hidden neurons are connected to all other neurons (Fig. 3A). We choose this configuration because it is widely used in neural networks and allowed recurrence in the connectivity of hidden units, although the architecture choice is not restrictive. All units are assigned reference \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${c}_{ik}^{\ast }$$\end{document}cik∗ value (selected at random from distribution P(cij) shown in Fig. 1), and all units except sensor units are assigned an objective mean value \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${m}_{i}^{\ast }=0$$\end{document}mi∗=0. During learning, the agent applies the rule in Equation 3 for adjusting its means and correlations to the assigned values. Each simulation step, the units of the Ising model are updated in a sequential random order using Glauber dynamics (Equation 1).Figure 3(A) Structure of the embodied neural controller for a model with N = 12 units and Nh = 6 hidden units. (B) Mountain Car environment: an under-powered car that must drive up a steep hill by balancing itself to gain momentum. (C) Acrobot environment: an agent has to balance a double pendulum to reach the high part of the environment.