Driven by the need for parallelizable hyperparameter optimization methods, this paper studies open loop search methods in the sense that the sequence is predetermined and can be generated before a single configuration is evaluated. Examples include grid search, uniform random search, low discrepancy sequences, and other sampling distributions. In particular, we propose the use of k-determinantal point processes in hyperparameter optimization via random search. Compared to conventional uniform random search where hyperparameter settings are sampled independently, a k-DPP promotes diversity. We describe an approach that transforms hyperparameter search spaces for efficient use with a k-DPP. In addition, we introduce a novel Metropolis-Hastings algorithm which can sample from kDPPs defined over spaces with a mixture of discrete and continuous dimensions. Our experiments show significant benefits over uniform random search in realistic scenarios with a limited budget for training supervised learners, whether in serial or parallel.
In this paper, the authors consider non-sequential (in the sense that many hyperparameter evaluations are done simultaneously) and uninformed (in the sense that the hyperparameter evaluations are chosen independent of the validation errors observed) hyperparameter search using determinantal point processes (DPPs). DPPs are probability distributions over subsets of a ground set with the property that subsets with more "diverse" elements have higher probability. Diverse here is defined using some similarity metric, often a kernel. Under the RBF kernel, the more diverse a set of vectors is, the closer the kernel matrix becomes to the identity matrix, and thus the larger the determinant (and therefore probability under the DPP) grows. The authors propose to do hyperparameter tuning by sampling a set of hyperparameter evaluations from a DPP with the RBF kernel.
Overall, I have a couple of concerns about novelty as well as the experimental evaluation for the authors to address. As the authors rightly point out, sampling hyperparameter values from a DPP is equivalent to sampling proportional to the posterior uncertainy of a Gaussian process, effectively leading to a pure exploration algorithm. As the authors additionally point out, such methods have been considered before, including methods that directly propose to batch Bayesian optimization by choosing a single exploitative point and sampling the remainder of the batch from a DPP (e.g., [Kathuria et al., 2016]). The default procedure for parallel BayesOpt used by SMAC [R2] is (I believe) also to choose a purely explorative batch. I am unconvinced by the argument that "while this can lead to easy parallelization within one iteration of Bayesian optimization, the overall algorithms are still sequential." These methods can typically be expanded to arbitrarily large batches and fully utilize all parallel hardware. Most implementations of batch Bayesian optimization in practice (SMAC and Spearmint as examples) will even start new jobs immediately as jobs finish -- these implementations do not wait for the entire batch to finish typically.
Additionally, while there has been some work extending GP-based BayesOpt to tree-based parameters [R3], at a minimum SMAC in particular is known well suited to the tree-based parameter search considered by the authors. I am not sure that I agree that TPE is state-of-the-art on these problems: SMAC typically does much better in my experience. 
Ultimately, my concern is that--considering these tools are open source and relatively stable software at this point--if DPP-only based hyperparameter optimization is truly better than the parallelization approach of SMAC, it should be straightforward enough to download SMAC and demonstrate this. If the argument that BayesOpt is somehow "still sequential" is true, then k-DPP-RBF should outperform these tools in terms of wall clock time to perform optimization, correct?
[R1] Kathuria, Tarun and Deshpande, Amit and Kohli, Pushmeet. Batched Gaussian Process Bandit Optimization via Determinantal Point Processes, 2016.
[R2] Several papers, see: http://www.cs.ubc.ca/labs/beta/Projects/SMAC/
[R3] Jenatton, R., Archambeau, C., González, J. and Seeger, M., 2017, July. Bayesian Optimization with Tree-structured Dependencies. In International Conference on Machine Learning (pp. 1655-1664).