Process-constrained batch Bayesian Optimisation
Prevailing batch Bayesian optimisation methods allow all control variables to be freely altered at each iteration. Real-world experiments, however, often have physical limitations making it time-consuming to alter all settings for each recommendation in a batch. This gives rise to a unique problem in BO: in a recommended batch, a set of variables that are expensive to experimentally change need to be fixed, while the remaining control variables can be varied. We formulate this as a process-constrained batch Bayesian optimisation problem. We propose two algorithms, pc-BO(basic) and pc-BO(nested). pc-BO(basic) is simpler but lacks convergence guarantee. In contrast pc-BO(nested) is slightly more complex, but admits convergence analysis. We show that the regret of pc-BO(nested) is sublinear. We demonstrate the performance of both pc-BO(basic) and pc-BO(nested) by optimising benchmark test functions, tuning hyper-parameters of the SVM classifier, optimising the heat-treatment process for an Al-Sc alloy to achieve target hardness, and optimising the short polymer fibre production process.
The paper presents a bayesian optimization framework in which samples can be collected in similarly constrained batches. In short, rather than selecting a single point to sample a label from (as in usual bayesian optimization) or selecting multiple points simultaneously (as in batch bayesian optimization), multiple samples in a batch can be collected in which one or more variables need to remain constant throughout the batch. This is motivated from experiments in which some parameter (e.g., temperature in an oven, geometry of an instrument) can be designed prior to sampling, and be reused in multiple experiments in a batch, but cannot be altered modified during a batch experiment.
The authors (a) propose this problem for the first time (b) propose two algorithms for solving it, (b) provide bounds on the regret for the most sophisticated of the two algorithms, and (c) present a comparison of the two algorithms on a variety of datasets. One experiment actually involves a metallurgical setup, in which the authors collaborated with researchers in their institution to design a metal alloy, an interdisciplinary effort that is commendable and should be encouraged.
My only concern is that the paper is somewhat dense. It is unlikely that someone unfamiliar with Bayesian optimization would be able to parse the paper. Even knowing about bayesian optimization, notation is not intoduced clearly and the algorithms are hard to follow. In particular:
-what is sigma in Algorithms 1, and 2. Is it the variance of the GP used in the UCB? 
-The activation fuction alpha^GB-UCB should be clearly and explicitly defined somewhere.
-The description of the inner loop and the outer loop in Algorithm 1 are not consistent with the exposition in the text: h and g are nowhere to be found in Algorithm 1. This seems to be because your replace the optimization with the GP-UCB-PE. It would be better if the link is made explicit.
-There is absolutely no description of how any optimization involving an activation function is performed. Do you use sampling? Gradient descent? Some combination of both?   These should be described, especially in how they where instantiated in the experiments.
-Is alpha different from alpha^GP-UCB in Alg. 1? how?
-In both algorithms 1 and 2, it is not clear how previously selected samples in a batch (k'\< k) are used to select the k-th sample in a batch, other than that they share a constrained variable.
 
All of these are central to understanding the algorithm. This is a pity, as the exposition is otherwise quite lucid.