title: Hyperband: Bandit-Based Configuration Evaluation for Hyperparameter Optimization
abstract: Performance of machine learning algorithms depends critically on identifying a good set of hyperparameters.  While recent approaches use Bayesian Optimization to adaptively select configurations, we focus on speeding up random search through adaptive resource allocation.  We present Hyperband,  a novel algorithm for hyperparameter optimization that is simple, flexible, and theoretically sound.  Hyperband is a principled early-stoppping method that adaptively allocates a predefined resource, e.g., iterations, data samples or number of features, to randomly sampled configurations.  We compare Hyperband with state-of-the-art Bayesian Optimization methods on several hyperparameter optimization problems.  We observe that Hyperband can provide over an order of magnitude speedups over competitors on a variety of neural network and kernel-based learning problems.  
authors: Lisha Li Kevin Jamieson Giulia DeSalvo Afshin Rostamizadeh Ameet Talwalkar
hIndex: Ameet Talwalkar 45