Noise-Aware Differentially Private Regression via Meta-Learning
<|reference_start|>Noise-Aware Differentially Private Regression via Meta-Learning: Many high-stakes applications require machine learning models that protect user privacy and provide well-calibrated, accurate predictions. While Differential Privacy (DP) is the gold standard for protecting user privacy, standard DP mechanisms typically significantly impair performance. One approach to mitigating this issue is pre-training models on simulated data before DP learning on the private data. In this work we go a step further, using simulated data to train a meta-learning model that combines the Convolutional Conditional Neural Process (ConvCNP) with an improved functional DP mechanism of Hall et al. [2013] yielding the DPConvCNP. DPConvCNP learns from simulated data how to map private data to a DP predictive model in one forward pass, and then provides accurate, well-calibrated predictions. We compare DPConvCNP with a DP Gaussian Process (GP) baseline with carefully tuned hyperparameters. The DPConvCNP outperforms the GP baseline, especially on non-Gaussian data, yet is much faster at test time and requires less tuning.<|reference_end|>
@article{räisä2024noise-aware,
    title={Noise-Aware Differentially Private Regression via Meta-Learning},
    author={Ossi R"ais"a, Stratis Markou, Matthew Ashman, Wessel P. Bruinsma,
  Marlon Tobaben, Antti Honkela, Richard E. Turner},
    journal={arXiv preprint arXiv:2406.08569},
    year={2024},
    archivePrefix={arXiv},
    eprint={2406.08569},
    primaryClass={cs.LG cs.CR stat.ML}
}