  Sample efficiency is one of the key factors when applying policy search to
real-world problems. In recent years, Bayesian Optimization (BO) has become
prominent in the field of robotics due to its sample efficiency and little
prior knowledge needed. However, one drawback of BO is its poor performance on
high-dimensional search spaces as it focuses on global search. In the policy
search setting, local optimization is typically sufficient as initial policies
are often available, e.g., via meta-learning, kinesthetic demonstrations or
sim-to-real approaches. In this paper, we propose to constrain the policy
search space to a sublevel-set of the Bayesian surrogate model's predictive
uncertainty. This simple yet effective way of constraining the policy update
enables BO to scale to high-dimensional spaces (>100) as well as reduces the
risk of damaging the system. We demonstrate the effectiveness of our approach
on a wide range of problems, including a motor skills task, adapting deep RL
agents to new reward signals and a sim-to-real task for an inverted pendulum
system.

Robotics