Ensuring the privacy of sensitive data used to train modern machine learning models is of paramount importance in many areas of practice. One approach to study these concerns is through the lens of differential privacy. In this framework, privacy guarantees are generally obtained by perturbing models in such a way that specifics of data used to train the model are made ambiguous. A particular instance of this approach is through a "teacher-student" model, wherein the teacher, who owns the sensitive data, provides the student with useful, but noisy, information, hopefully allowing the student model to perform well on a given task without access to particular features of the sensitive data. Because stronger privacy guarantees generally involve more significant noising on the part of the teacher, deploying existing frameworks fundamentally involves a trade-off between utility and privacy guarantee. One of the most important techniques used in previous work involves an ensemble of teacher models, which return information to a student based on a noisy voting procedure. In this work, we propose a novel voting mechanism, which we call an Immutable Noisy ArgMax, that, under certain conditions, can bear very large random noising from the teacher without affecting the useful information transferred to the student. Our mechanisms improve over the state-of-the-art methods on all measures, and scale to larger tasks with both higher utility and stronger privacy ( ≈ 0).
This paper studies the teacher ensembles setting for differentially private learning. In this setting, each teacher holds part of the training set and trains a local model. The student uses unlabeled examples to query teacher model. Then the student trains a model from scratch using the examples labeled by teachers.
In order to make the labeling process differentially private, previous work uses noisy argmax mechanism. Each class of label is assigned with a count number. The student first queries the same example to multiple teachers. To guarantee differential privacy, the counts are perturbed by noise before releasing. Then, because of the post-processing property of differential privacy, the argmax operator on such noisy counts are still differentially private.
This paper proposes to add a constant c to the largest count before perturbing and releasing the counts. The authors argue this would improve the accuracy of the noisy argmax operator and yield the same privacy loss as previous approach. However, adding a constant c would increase the sensitivity and therefore degenerates the privacy guarantee. The added noise cannot guarantee the privacy if all others are the same as previous work. To see this clearer, for example, if c=0, then one sample point can at most change the count by 1. If c>0, then one sample point can change the count by 1+c. Because of this, the proposed method cannot guarantee the amount of differential privacy as the paper claimed.