QUERY-EFFICIENT HARD-LABEL BLACK-BOX ATTACK: AN OPTIMIZATION-BASED APPROACH
We study the problem of attacking machine learning models in the hard-label black-box setting, where no model information is revealed except that the attacker can make queries to probe the corresponding hard-label decisions. This is a very challenging problem since the direct extension of state-of-the-art white-box attacks (e.g., C&W or PGD) to the hard-label black-box setting will require minimizing a non-continuous step function, which is combinatorial and cannot be solved by a gradient-based optimizer. The only two current approaches are based on random walk on the boundary (Brendel et al., 2017) and random trials to evaluate the loss function (Ilyas et al., 2018), which require lots of queries and lacks convergence guarantees. We propose a novel way to formulate the hard-label black-box attack as a real-valued optimization problem which is usually continuous and can be solved by any zeroth order optimization algorithm, such as randomized gradientfree method (Nesterov & Spokoiny, 2017). We demonstrate that our proposed method outperforms the previous stochastic approaches to attacking convolutional neural networks on MNIST, CIFAR, and ImageNet datasets. More interestingly, the proposed algorithm can also be used to attack other discrete and non-continuous machine learning models, such as Gradient Boosting Decision Trees.
This paper addresses black-box classifier attacks in the “hard-label” setting, meaning that the only information the attacker has access to is single top-1 label predictions. Relative to even the standard black-box setting where the attacker has access to the per-class logits or probabilities, this setting is difficult as it makes the optimization landscape non-smooth. The proposed approach reformulates the optimization problem such that the outer-loop optimizes the direction using approximate gradients, and the inner-loop estimates the distance to the nearest attack in a given direction. The results show that the proposed approach successfully finds both untargeted and targeted adversarial examples for classifiers of various image datasets (including ImageNet), usually with substantially better query-efficiency and better final results (lower distance and/or higher success rate) than competing methods.
=====================================
Pros:
Very well-written and readable paper with good background and context for those (like me) who don’t closely follow the literature on adversarial attacks. Figs. 1-3 are nice visual aids for understanding the problem and optimization landscape.
Novel formulation and approach that appears to be well-motivated from the literature on randomized gradient-free search methods. Novel theoretical analysis in Appendix that generalizes prior work to approximations (although, see notes below).
Good empirical results showing that the method is capable of query-efficiently finding attacks of classifiers on real-world datasets including ImageNet. Also shows that the model needn’t be differentiable to be subject to such attacks by demonstrating the approach on a decision-tree based classifier. Appears to compare to and usually outperform appropriate baselines from prior work (though I’m not very familiar with the literature here).
=====================================
Cons/questions/suggestions/nitpicks:
Eq 4-5: why \texttt argmin? Inconsistent with other min/maxes.
Eq 4-5: Though I understood the intention, I think the equations are incorrect as written: argmin_{\lambda} { F(\lambda) } of a binary-valued function F would produce the set of all \lambdas that make F=0, rather than the smallest \lambda that makes F=1. I think it should be something like:
min_{\lambda>0} {\lambda}
s.t. f(x_0+\lambda \theta/||\theta||) != y_0
Sec 3.1: why call the first search “fine-grained”? Isn’t the binary search more fine-grained? I’d suggest changing it to “coarse-grained” unless I’m misunderstanding something.
Algorithm 2: it would be nice if this included all the tricks described as “implementation details” in the paragraph right before Sec. 4 -- e.g. averaging over multiple sampled directions u_t and line search to choose the step size \eta. These seem important and not necessarily obvious to me.
Algorithm 2: it could be interesting to show how performance varies with number of sampled directions per step u_t.
Sec: 4.1.2: why might your algorithm perform worse than boundary-attack on targeted attacks for CIFAR classifiers? Would like to have seen at least a hypothesis on this.
Sec 6.3 Theorem 1: I think the theorem statement is a bit imprecise. There is an abuse of big-O notation here -- O(f(n)) is a set, not a quantity, so statements such as \epsilon ~ O(...) and \beta <= O(...) and “at most O(...)” are not well-defined (though common in informal settings) and the latter two are redundant given the meaning of O as an upper bound. The original theorem from [Nesterov & Spokoiny 2017] that this Theorem 1 would generalize doesn’t rely on big-O notation -- I think following the same conventions here might improve the theorem and proof.
=====================================
Overall, this is a good paper with nice exposition, addressing a challenging but practically useful problem setting and proposing a novel and well-motivated approach with strong empirical results.