Contamination Attacks and Mitigation in Multi-Party Machine Learning
Machine learning is data hungry; the more data a model has access to in training, the more likely it is to perform well at inference time. Distinct parties may want to combine their local data to gain the benefits of a model trained on a large corpus of data. We consider such a case: parties get access to the model trained on their joint data but do not see each others individual datasets. We show that one needs to be careful when using this multi-party model since a potentially malicious party can taint the model by providing contaminated data. We then show how adversarial training can defend against such attacks by preventing the model from learning trends specific to individual parties data, thereby also guaranteeing party-level membership privacy.
**Review updated post feedback** Thank you to the authors for the clarifications included in their feedback, I updated my review accordingly below.  This submission sets out to study poisoning attacks in the context of collaborative ML wherein the attacker is a malicious participant that poisons some of the training data to encode a specific correlation between input attributes and output labels. The submission then considers the robustness of this attack in the face of adversarial training (in the sense of GANs).  The threat model's presentation can be improved:  * The feedback clarified which specific learning setting is considered in this submission (collaborative but not distributed).  * Furthermore, it would be beneficial to expand upon the distinction between contamination attacks and other existing poisoning attacks. My suggestion would be to stress that contamination attacks do not require a specific trigger to be added to the input presented at test time. Examples relating contamination attacks to fairness issues with respect to some of the input attributes are helpful as well to help position this paper within the existing body of literature.  * The submission has ties to work published in security conferences on property inference [a] and [14]. I think there are differences but the present submission could better articulate them and perhaps add some more detailed comparison points than what is currently included in the related work paragraph.  Moving beyond the lack of clarity regarding the threat model, adversarial capabilities assumed in the rest of the manuscript and required to mount the attack could be made more explicit. This aspect was clarified in the author feedback. Indeed, notation potentially misleads readers into thinking that both the ManipulateData and TrainModel procedures are part of the attacker pipelines.   It would have been interesting to see a discussion of how the adversary chooses contaminated attributes and how contamination accuracy relates to the existence of sensitive attributes in a given dataset.  In Section 5.2, it is proposed to apply adversarial training to defend against contamination attacks. However, the evaluation does not consider any adaptive attack strategies to answer the following question: how could adversaries adapt their strategy if they were aware of the defense being deployed? The author feedback addresses this point using guarantees of the mini-max game convergence. However, given that game convergence relies (empirically) on the choice of constant c, it would be beneficial to expand upon this in the manuscript or supplementary material.   Additional minor details: * Abstract uses vague terms. * What is g’s “own objective function” at l205 of p6? * Membership inference attacks are only discussed in the very last paragraph of the manuscript. They could be introduced earlier to help readers position the paper in existing literature.  * Typo line 208: "memership inference attack"  [a] G. Ateniese, L. V. Mancini, A. Spognardi, A. Villani, D. Vitali, and G. Felici. Hacking smart machines with smarter ones: How to extract meaningful data from machine learning classifiers. International Journal of Security and Networks, 2015.