Local Differential Privacy for Evolving Data
There are now several large scale deployments of differential privacy used to collect statistical information about users. However, these deployments periodically recollect the data and recompute the statistics using algorithms designed for a single use. As a result, these systems do not provide meaningful privacy guarantees over long time scales. Moreover, existing techniques to mitigate this effect do not apply in the "local model" of differential privacy that these systems use. In this paper, we introduce a new technique for local differential privacy that makes it possible to maintain up-to-date statistics over time, with privacy guarantees that degrade only in the number of changes in the underlying distribution rather than the number of collection periods. We use our technique for tracking a changing statistic in the setting where users are partitioned into an unknown collection of groups, and at every time period each user draws a single bit from a common (but changing) group-specific distribution. We also provide an application to frequency and heavy-hitter estimation.
This paper mainly considered streaming mean estimation problem under local differential privacy model. Though there has been lots of work about locally private mean estimation problem, utility guarantees of naïve implementation for previous methods in the streaming model will degrade with time period t at least in order O(\sqrt{t}), because of composition theorem. The main contribution of this paper was proposing an interactive private algorithm that achieved utility only in O(\ln T) order under reasonable settings, and its idea kind of resembled sparse vector technique in DP model. Further, authors also showed a direct application of their technique for streaming heavy hitter problem.      Overall, this paper is well written, and I personally quite like its idea. Given fixed privacy budget, utility guarantees of common private algorithms usually degrade with time t in polynomial order, which is a well-known problem in privacy literature. This paper makes some sound assumptions about the underlying estimation problem, and obtains an O(\ln T) order performance guarantee, which advances existing techniques and results.   However, I feel kind of confused about some parts in the paper, and I put them in the below “Cons” part.   Pros: 1. Design a novel locally private algorithm that can achieve O(ln T) order performance in the streaming estimation problem under reasonable assumptions; 2. Apply above technique to common heavy hitter problem and obtain corresponding guarantees.   Cons: 1. In the proof of lemma 7.6 (line 454), how do we obtain this inequality through the fact “user i sets VoteYes_i^t’ as true”?  What if b^* equals -1, and 2^{log T – b^*} divides t? In this case, VoteYes_i^t’ is also set as yes, but we don’t have above inequality (i.e. line 454); 2. Assumption 4.2 seems to be a little restricted. What if this assumption doesn’t hold? For example, the size of smallest subgroup is in order O(n^{1/4});  3. In line 261, authors assume d >> n for heavy hitter problem. I know it is a common parameter regime, but I wonder whether d >> n is a necessary condition like assumption 4.2. What if d is not so large as specified in line 297?   Other minor comments: 1. In line 159, \mu^r should be \mu^r = 1/n * (|S_1|*\mu_1^r+ … + |S_m|* \mu_m^r); 2. In line 198, in the definition of \tao_b, constant 10 should be 12 according to the appendix; 3. In line 5 of the algorithm 5 (and other similar places), the notation of noise level b is misleading with the subscript of \tao_b; 4. In line 19-21 of the algorithm 5, it seems it doesn’t play any role in the algorithm, and could be deleted; 5. Line 386 is reduplicative with previous sentence; 6. In the bottom inequality of page 10, it should be a^t in the bracket, not (a^t, \tilde{p}^t); 7. In the second equation in line 392, authors omitted a^t in the conditional probability for \tilde{p}^t.   Update: Thanks for authors' detailed responses. For my first concern, I just misunderstood the decide condition. For the 4th comment in "minor comments" part, there is a typo  and algorithm 5 should be algorithm 1. Overall, it is a good paper.