Black-box adversarial attacks require a large number of attempts before finding successful adversarial examples that are visually indistinguishable from the original input. Current approaches relying on substitute model training, gradient estimation or genetic algorithms often require an excessive number of queries. Therefore, they are not suitable for real-world systems where the maximum query number is limited due to cost. We propose a query-efficient black-box attack which uses Bayesian optimisation in combination with Bayesian model selection to optimise over the adversarial perturbation and the optimal degree of search space dimension reduction. We demonstrate empirically that our method can achieve comparable success rates with 2-5 times fewer queries compared to previous stateof-the-art black-box attacks.
This paper studied the problem of black-box adversarial attack generation by leveraging Bayesian optimization (BO).
Merits of this paper:
1) The combination of BO and dimension reduction, which makes BO more efficient under a low-dimensional space. 
2) Good experiment results.
Comments/questions about this paper:
1) Comment on "Finally, to the best of our knowledge, the only prior work that uses Bayesian optimisation is a workshop paper by...". BO was also used for generating black-box adversarial examples at https://arxiv.org/pdf/1907.11684.pdf
This is a missing related work, and please elaborate on the differences. 
2) The presentation of the proposed algorithm is not clear. Please explicitly state the acquisition function. And how to tune the hyperparameter in the acquisition function? What decoder is used in experiments? Have authors tested the sensitivity of the decoder (not reduced dimension)?  
3) In experiments, the authors mentioned "we randomly select 3 correctly classified images for each class from CIFAR10 test data which sums up to 27 CIFAR10 images, and randomly select 7 correctly classified images from MNIST test data."
I feel that the number of tested images is not sufficient. How about conducting experiments on a large number of tested images for untargeted attack?
4) In Table 1, what does 0,0,0, mean in ZOO? 
5) It is known that BO has itself parameter to tune, and is not computationally efficient. It might be good to show the computation efficiency of BO for different reduced dimensions together with the corresponding attack performance. 
6) The convergence of BO is usually not stable. However, Figure 3 shows that BO converges very smoothly in terms of ASR. Could authors also show the loss value of using BO-attacks against iteration numbers? 
Meanwhile, in Figure 3 is the best ASR  (up to the current query counts) reported or the ASR at the current query number?
Based on the aforementioned questions, my initial rating is weak reject. 
############## Post-feedback ################
Thanks for the response. Most of my questions have been addressed. Thus, I increased my score to 6. 
I suggested to have a clearer presentation on the possible pros and cons of BO in attack generation, e.g., making a comparison between BO and other methods in both query efficiency and computation efficiency.