Closed-Form Bounds for DP-SGD against Record-level Inference
<|reference_start|>Closed-Form Bounds for DP-SGD against Record-level Inference: Machine learning models trained with differentially-private (DP) algorithms such as DP-SGD enjoy resilience against a wide range of privacy attacks. Although it is possible to derive bounds for some attacks based solely on an $(\varepsilon,\delta)$-DP guarantee, meaningful bounds require a small enough privacy budget (i.e., injecting a large amount of noise), which results in a large loss in utility. This paper presents a new approach to evaluate the privacy of machine learning models against specific record-level threats, such as membership and attribute inference, without the indirection through DP. We focus on the popular DP-SGD algorithm, and derive simple closed-form bounds. Our proofs model DP-SGD as an information theoretic channel whose inputs are the secrets that an attacker wants to infer (e.g., membership of a data record) and whose outputs are the intermediate model parameters produced by iterative optimization. We obtain bounds for membership inference that match state-of-the-art techniques, whilst being orders of magnitude faster to compute. Additionally, we present a novel data-dependent bound against attribute inference. Our results provide a direct, interpretable, and practical way to evaluate the privacy of trained models against specific inference threats without sacrificing utility.<|reference_end|>
@article{cherubin2024closed-form,
    title={Closed-Form Bounds for DP-SGD against Record-level Inference},
    author={Giovanni Cherubin, Boris K"opf, Andrew Paverd, Shruti Tople, Lukas
  Wutschitz, Santiago Zanella-B'eguelin},
    journal={arXiv preprint arXiv:2402.14397},
    year={2024},
    archivePrefix={arXiv},
    eprint={2402.14397},
    primaryClass={cs.CR cs.LG}
}