We present a probabilistic framework for studying adversarial attacks on discrete data. Based on this framework, we derive a perturbation-based method, Greedy Attack, and a scalable learning-based method, Gumbel Attack, that illustrate various tradeoffs in the design of attacks. We demonstrate the effectiveness of these methods using both quantitative metrics and human evaluation on various stateof-the-art models for text classification, including a word-based CNN, a characterbased CNN and an LSTM. As an example of our results, we show that the accuracy of character-based convolutional networks drops to the level of random selection by modifying only five characters through Greedy Attack.
This paper addresses the problem of generating adversarial examples for discrete domains like text. They propose two simple techniques:
1) Greedy: two stage process- first stage involves finding the k words in the sentence/paragraph to perturb and second step changes the word in the positions identified in step 1.
2) Gumbel: first approach amortized over datasets where first and second steps are parametrized and learned over the dataset with the loss being the probability of flipping the decision.
Specifically, for the Gumbel approach, the authors use the non-differentiable top-k-argmax output to train the module in the second step which is not ideal and it would be better to train both first and second steps jointly in an end-to-end differentiable manner.
The results show that Greedy approach is able to significantly affect the accuracy of the systems compared to other adversarial baselines. Mturk evaluation shows that for tasks like sentiment analysis, humans weren't as confused as the systems were when the selected words were changed which is encouraging. However, the Gumbel method performs poorly compared to other baselines.
Moreover, a thorough analysis of why Greedy is doing better than some gradient based adversarial attacks is needed in the paper because it is unclear what is causing their greedy approach to perform well; is it the two-stage nature of the process?
My major gripe with the paper is that it is egregiously difficult to read in parts and is poorly written. There are dangling conditional bars in many equations (5, 7, Greedy attack etc.), unclear "expectation (E)" signs and many other confusing notational choices which make the math difficult to parse. I am not even sure if those equations are correctly conveying the idea they are meant to convey. I found  the algorithms to be more clearly written and realize that the text in the models and equations is unnecessarily complicated. The argument about approximation to the objective by considering the i positions independently is not convincing and their is nothing in the paper to show if the assumption is reasonable.