Adversarial Attacks on Stochastic Bandits
We study adversarial attacks that manipulate the reward signals to control the actions chosen by a stochastic multi-armed bandit algorithm. We propose the first attack against two popular bandit algorithms: ✏-greedy and UCB, without knowledge of the mean rewards. The attacker is able to spend only logarithmic effort, multiplied by a problem-specific parameter that becomes smaller as the bandit problem gets easier to attack. The result means the attacker can easily hijack the behavior of the bandit algorithm to promote or obstruct certain actions, say, a particular medical treatment. As bandits are seeing increasingly wide use in practice, our study exposes a significant security threat.
This paper presents adversarial attacks against multi-arm bandit algorithms e-greedy and UCB. This paper is very well written;                 the problem is well-motivated; the approach is sound and well explained through the guidance from an oracle approach in an                 less realistic setting to the real setting. I enjoy reading this paper.                 I list some of the aspects that can be improved below and hope they can help the authors to improve the paper. But, again, I                think the paper is publishable even in the current version. First, it would be better if the threat model can be further                 justified, i.e., why the attacker can manipulate the rewards arbitrarily? It would be good if the authors can map the attacker                to real settings.                 Second, I don't find how the attacks can be mitigated if Bob is aware of the existence of the attacker. This would be an                 interesting discussion to see to what can be an equilibrium between the attacker and the defender.                 Third, which is related to the above aspects, the authors only consider two bandit algorithms, leaving a large space of other                bandit algorithms not discussed. Thus, it is unclear whether Bob can simply choose another algorithm to defend against the                attacks? Would there be any universal results that can be said about the attacker without relying on the concrete bandit                 algorithm chosen by the defender? The authors admit that the work is still in progress, and many more bandit algorithms will                be discussed; I hope, in the future versions, there could be some general conclusions presented in par of these concrete                bandit algorithms, which might provide more tight results.