In Fig. 7A–C we can observe the behaviour of a Mountain Car agent with Nh = 64 for β = {0.75, 1, 1.2}, respectively. We observe that for values of β lower than the operating temperature, the agents are not able to reach the top of the mountain. On the other hand, when β is higher, the agents present more ‘rigid’ trajectories going from one mountain peak to the other. At β = 1 the agent is able to reach the top of the mountain (note that the peaks of the mountain are located at x = −π/2 and x = π/6) while displaying larger behavioural diversity. Similarly, in Fig. 7D–F, we observe that the Acrobot agent with Nh = 64 at β = 1 displays a diverse range of behaviours, being able to reach the top of the plane while, when β is lowered or increased, it drifts to other behavioural modes in less diverse regimes. Although only one agent is represented for each environment, the results of Fig. 7 are similar in all agents and sizes.Figure 7Transition in the behavioural regime of the agents with Nh = 64. We show the behaviour of two individual agents with different values of β for the Mountain Car (A–C) and Acrobot (D–F) embodiments. We observe that β = 1 is a transition point between two modes of behaviour in both agents.