  In addition to their undisputed success in solving classical optimization
problems, neuroevolutionary and population-based algorithms have become an
alternative to standard reinforcement learning methods. However, evolutionary
methods often lack the sample efficiency of standard value-based methods that
leverage gathered state and value experience. If reinforcement learning for
real-world problems with significant resource cost is considered, sample
efficiency is essential. The enhancement of evolutionary algorithms with
experience exploiting methods is thus desired and promises valuable insights.
This work presents a hybrid algorithm that combines topology-changing
neuroevolutionary optimization with value-based reinforcement learning. We
illustrate how the behavior of policies can be used to create distance and loss
functions, which benefit from stored experiences and calculated state values.
They allow us to model behavior and perform a directed search in the behavior
space by gradient-free evolutionary algorithms and surrogate-based
optimization. For this purpose, we consolidate different methods to generate
and optimize agent policies, creating a diverse population. We exemplify the
performance of our algorithm on standard benchmarks and a purpose-built
real-world problem. Our results indicate that combining methods can enhance the
sample efficiency and learning speed for evolutionary approaches.

Neural and Evolutionary Computing;Artificial Intelligence