  Deep Reinforcement Learning (RL) is proven powerful for decision making in
simulated environments. However, training deep RL model is challenging in real
world applications such as production-scale health-care or recommender systems
because of the expensiveness of interaction and limitation of budget at
deployment. One aspect of the data inefficiency comes from the expensive
hyper-parameter tuning when optimizing deep neural networks. We propose
Adaptive Behavior Policy Sharing (ABPS), a data-efficient training algorithm
that allows sharing of experience collected by behavior policy that is
adaptively selected from a pool of agents trained with an ensemble of
hyper-parameters. We further extend ABPS to evolve hyper-parameters during
training by hybridizing ABPS with an adapted version of Population Based
Training (ABPS-PBT). We conduct experiments with multiple Atari games with up
to 16 hyper-parameter/architecture setups. ABPS achieves superior overall
performance, reduced variance on top 25% agents, and equivalent performance on
the best agent compared to conventional hyper-parameter tuning with independent
training, even though ABPS only requires the same number of environmental
interactions as training a single agent. We also show that ABPS-PBT further
improves the convergence speed and reduces the variance.

Machine Learning;Artificial Intelligence;Machine Learning