Enhancing Learning with Label Differential Privacy by Vector Approximation
<|reference_start|>Enhancing Learning with Label Differential Privacy by Vector Approximation: Label differential privacy (DP) is a framework that protects the privacy of labels in training datasets, while the feature vectors are public. Existing approaches protect the privacy of labels by flipping them randomly, and then train a model to make the output approximate the privatized label. However, as the number of classes $K$ increases, stronger randomization is needed, thus the performances of these methods become significantly worse. In this paper, we propose a vector approximation approach, which is easy to implement and introduces little additional computational overhead. Instead of flipping each label into a single scalar, our method converts each label into a random vector with $K$ components, whose expectations reflect class conditional probabilities. Intuitively, vector approximation retains more information than scalar labels. A brief theoretical analysis shows that the performance of our method only decays slightly with $K$. Finally, we conduct experiments on both synthesized and real datasets, which validate our theoretical analysis as well as the practical performance of our method.<|reference_end|>
@article{zhao2024enhancing,
    title={Enhancing Learning with Label Differential Privacy by Vector
  Approximation},
    author={Puning Zhao, Rongfei Fan, Huiwen Wu, Qingming Li, Jiafei Wu, Zhe Liu},
    journal={arXiv preprint arXiv:2405.15150},
    year={2024},
    archivePrefix={arXiv},
    eprint={2405.15150},
    primaryClass={cs.LG}
}