Bayesian optimization - One run with 100 iterations vs ten runs with ten iterations I was wondering whether it makes more sense to run Bayesian Optimization let's say ones with 100 iterations or ten times with ten iterations each. Which one should I favor?
I would assume that the first is better since the probabilistic model of the objective function gets better?
I'm using Bayesian Optimization to tune multiple hyperparameters of a Random Forest model - max_depth, max_features, min_samples_leaf, and min_samples_split. Currently, I'm using the library pyGPGO with a Gaussian Process and expected improvement as an acquisition function.

We can't say without knowing your specific problem.
I would suggest you first run $10\times 10$. Then assess how variable your 10 results are (e.g., in terms of the variability of predictions on a holdout dataset). If this variability is "low", you can presumably trust 10 iterations. If this variability is "high", then I would not trust any one of the 10 iterations (I would probably trust an ensemble of the 10 instances more), and I would repeat with $10\times 100$, or whatever number is feasible, and iterate until the results stabilize.
You will need to consult your domain knowledge to operationalize "low" and "high".
