Data Poisoning Attacks on Factorization-Based Collaborative Filtering
Recommendation and collaborative filtering systems are important in modern information and e-commerce applications. As these systems are becoming increasingly popular in the industry, their outputs could affect business decision making, introducing incentives for an adversarial party to compromise the availability or integrity of such systems. We introduce a data poisoning attack on collaborative filtering systems. We demonstrate how a powerful attacker with full knowledge of the learner can generate malicious data so as to maximize his/her malicious objectives, while at the same time mimicking normal user behavior to avoid being detected. While the complete knowledge assumption seems extreme, it enables a robust assessment of the vulnerability of collaborative filtering schemes to highly motivated attacks. We present efficient solutions for two popular factorizationbased collaborative filtering algorithms: the alternative minimization formulation and the nuclear norm minimization method. Finally, we test the effectiveness of our proposed algorithms on real-world data and discuss potential defensive strategies.
This paper investigates how attackers can inject malicious data into the training dataset of a recommendation system, in order to either maximise the error of the system, or increase/decrease the popularity of a subset of certain items. Specifically, they design a projective gradient ascent method (PGA) that optimises the attack strategy. They also propose a technique based on stochastic gradient Langevin dynamics (SGLD) optimization that can generate malicious data mimicking normal users. Experimental results show that PGA and SGLD are more effective than strawman uniform attacks.
The paper explores an important topic – adversarial machine learning. While the paper contributes interesting results, it seems slightly lacking in novelty/depth.  In general, the paper is well presented. Both the attack models and strategies are clearly derived and explained. It is indeed important to have this kind of analysis to fully understand the vulnerability of collaborative filtering schemes. New connections are made (to SGLD).  A main concern revolves around novelty/impact. Researchers have achieved numerous attacks on learners now (with most key references cited appropriately in the paper), demonstrating that the vulnerability of learning to poisoning is fundamentally well understood. Attacks tend to be formulated in the same way, as an optimisation on the prediction score function (or learning map; or the composition of the two); combined with some form of gradient descent, leaving just the calculation of (sub)gradients. It is interesting to follow through with this (for a learner of choice) if the attack has some closed form solution that can be defended against, or in some way provides a surprising conclusion – not the case here.  Formulating the different kinds of attacks on learners (availability, integrity, etc.) is not novel as a contribution. The paper should probably cite Barreno et al. (how secure is machine learning circa 2006 or so), or the papers that followed it up by the same authors.  Attacking the iterative fitting process (not just a global solution) appears more novel. However the calculations for the attacks, while very involved and non-trivial are not novel in themselves. For example exploiting KKT conditions (line 165) has been done by e.g. Mei and Zhu (AISTATS 2015) in the context of attacking LDA.  Mimicry attacks are well known e.g. Tan et al. (2002) were one of the earliest (applied to intrusion detection); Wagner & Soto (2002); Wang et al. (2006) the Anagram system. Formulating them as an optimisation is neat. The hybrid attack doesn't seem as well motivated as the pure strategies.  A point missing is a discussion on the feasibility of the attacks. Even if the attacker has complete knowledge of the target system, the overhead of computing the gradient does not seem to be trivial. In addition, the experimental results suggest that when the purpose is to maximise the system error, there is not much difference between PGA/SGLD and uniform attacks, until the percentage of poisoning samples reaches 10% or even 15% (Fig. 1 (a) (b), Fig. 2 (a) (b)). It isn't clear that the overhead (in computation, but more importantly sophistication) outweighs the gain. In practice, is it practical for attackers to insert that much malicious data? This is important, in the context of the next comment: that the uniform attack is a strawman.  Another problem with the experiment is that considering uniform attacks, both ratings and items are randomly chosen. This doesn’t make sense when the purpose is to boost/reduce the popularity of certain specific items – would a realistic baseline not bias sampling towards such items (or perform something like a dictionary attack).  Updating from author feedback: Thankyou for the response. While my position on novelty remains substantively the same, I would agree with the authors that the typical stance in cybersecurity is that worst-case attacks are indeed of interest. For one, "no security through obscurity" (we cannot guarantee robustness by assuming the attacker is unaware of training data) but also, the worst-case bounds the average case. That said, the most powerful attacks are those that demonstrate influence over a target system while leveraging minimal capabilities.