PATE-GAN: GENERATING SYNTHETIC DATA WITH DIFFERENTIAL PRIVACY GUARANTEES
Machine learning has the potential to assist many communities in using the large datasets that are becoming more and more available. Unfortunately, much of that potential is not being realized because it would require sharing data in a way that compromises privacy. In this paper, we investigate a method for ensuring (differential) privacy of the generator of the Generative Adversarial Nets (GAN) framework. The resulting model can be used for generating synthetic data on which algorithms can be trained and validated, and on which competitions can be conducted, without compromising the privacy of the original dataset. Our method modifies the Private Aggregation of Teacher Ensembles (PATE) framework and applies it to GANs. Our modified framework (which we call PATE-GAN) allows us to tightly bound the influence of any individual sample on the model, resulting in tight differential privacy guarantees and thus an improved performance over models with the same guarantees. We also look at measuring the quality of synthetic data from a new angle; we assert that for the synthetic data to be useful for machine learning researchers, the relative performance of two algorithms (trained and tested) on the synthetic dataset should be the same as their relative performance (when trained and tested) on the original dataset. Our experiments, on various datasets, demonstrate that PATE-GAN consistently outperforms the stateof-the-art method with respect to this and other notions of synthetic data quality.
[Post revision update] The authors' comments addressed my concerns, especially on the experiment side. I changed the score.
This paper applies the PATE framework to GAN, and evaluates the quality of the generated data with some predictive tasks. The experimental results on some real datasets show that the proposed algorithm outperforms DPGAN, and the generated synthetic data is quite useful in comparison with real data.
The presentation is clear and easy to follow. However, I think the paper needs to be improved in its novelty, and the techniques and experiments need to be more thorough.
More details:
- It might be necessary to consider using Gaussian noise[24] in replace of the Laplace noise, which, according to [24], would improve privacy and accuracy.
- This paper:
“Privacy-preserving generative deep neural networks support clinical data sharing” by Brett K. Beaulieu-Jones, Zhiwei Steven Wu, Chris Williams, Casey S. Greene
seems quite relevant. If so, you may want to add some discussion in the related work section or compare with their result.
- The last paragraph of the related works section mentioned some related work with shortcomings as working only on low-dimensional data and features of specific types, yet the experiments are also mostly done on low-dimensional datasets. I think it would be better to do a thorough evaluation on data of different kinds, such as image data. 
- If the two evaluation metrics for private GAN is considered an important contribution of the paper, it might be better to make it a separate section and elaborate more on the motivation and method.  
- It might be better to move some details (for example, instead of presenting the results of the 12 predictive models, presenting only the average, as it’s not very important how each of them performs) of the credit card fraud detection dataset to the appendix and bring the results of the other datasets to the main body.