Extremal Mechanisms for Local Differential Privacy
Local differential privacy has recently surfaced as a strong measure of privacy in contexts where personal information remains private even from data analysts. Working in a setting where the data providers and data analysts want to maximize the utility of statistical inferences performed on the released data, we study the fundamental tradeoff between local differential privacy and information theoretic utility functions. We introduce a family of extremal privatization mechanisms, which we call staircase mechanisms, and prove that it contains the optimal privatization mechanism that maximizes utility. We further show that for all information theoretic utility functions studied in this paper, maximizing utility is equivalent to solving a linear program, the outcome of which is the optimal staircase mechanism. However, solving this linear program can be computationally expensive since it has a number of variables that is exponential in the data size. To account for this, we show that two simple staircase mechanisms, the binary and randomized response mechanisms, are universally optimal in the high and low privacy regimes, respectively, and well approximate the intermediate regime.
The paper claims to make a contribution to local differential privacy.  Unfortunately, I didn't sufficiently understand the text to confidently interprete and summarize the content.The quality of the paper was not sufficient to let me understand its main message.  In particular, the writing is quite vague and unclear.The paper describes some related work, but not being able to interprete the fine details of the contribution, I couldn't assess the originality and significance.The following are detailed comments in order of occurrence in the text:The abstract is a bit vague wrt. the topic and contribution of the paper.  It says "We address a general problem of utility maximization under local differential privacy." but doesn't specify what this "general problem of utility maximization" is.   The rest of the abstract is therefore over the top of my head upon first reading.The authors define local privacy as when "individuals disclose their personal information (e.g. voluntarily on social network sites).", but if individuals disclose data to the world, what is then the problem of privacy?The authors discuss that "differential privacy has been extended to local privacy" and use in their formal definition "privatized views Y_i" without explaining what are privatized views (remember from the previous comment that the data is already public).  These "view" (usually inputs to the viewer) are also called "outputs" in the next sentence.  In this definition, it is also not clear why exactly two distributions (P_0 and P_1) are considered.  The sentence "We consider Q’s satisfying the privacy conditions," is unclear in not specifying what is the "privacy condition" (is it the differential privacy discussed later in the definition?).  The formula of the definition looks a common one, so I guess the problem is mainly in the formalization and wording.This definition of differential privacy uses Q for a conditional probability distribution, where the conditions are on one single x_i and n-1 different y_j's.  Formula (2) uses Q with conditioning on n x_i's.  The number of conditionings is the same, but it is unclear how they map to each other.  Moreover, the "privatization" formulation suggests that the x_i and y_j are not necessarily elements of the same domain.The introduction further states that the power to discriminate data from P_0 from data from P_1 depends on the marginals M_\nu.  However, it seems there are other ways to discriminate, e.g. by just considering the distribution of the values of X_i for some i (everyone knows at least one X_i).Line 110: "Q the minimized the error" -> "Q that minimizes the error" It is unclear why Equation (4) introduces f-divergence and then (5) specializes to KL-divergence (which the paper says is a special case).Line 175: and marginal of the privatized output -> ungrammaticalSection 2 starts saying "We first give a formal definition of the staircase mechanisms," but I don't find such formal definition before theorem 2.1.  Theorem 2.1 is not proved in the main paper and does not refer to the supplementary material.  The proof in the supplementary material seems to use staircase notion (which as I said don't seem to be provided before that point).After this point, the technical content gets increasingly confusing.The paper has an usual structure, has no conclusion section, does not outline the structure at the end of the introduction.The paper does not provide experimental results, examples, illustrations, figures, or other attempts at making the content somewhat easier to digest.-------------------------------------------Additional comments after the author's rebuttal.W.r.t. Comment 3, Authors say: "the authors say: We provide in lines 121-157, a summary of our contributions in the context of existing literature."I reread now this part, and try to comment based on what I understand from the paper, the discussion with other reviewers and the author's response:* Line 122: In fact, the authors are not solving an optimization problem under local differential privacy constraints, but they are optimizing privatization mechanisms under local differential privacy constraints.  Taking the sentence literally, one (of the several) possible semantics for this sentence in isolation would be that we are given a privatized view and the data analyst should try to find the best possible value for some (model) parameter.  The second part of the sentence (resulting in a family of optimal mechanisms) is inconsistent with the first part.  * Terms such as "large class", "family", "important", ... are rather vague, and will need to be instantiated to the concrete contribution at some point (which does not happen in the part 121-157).  It is (especially here) unclear why "optimization problem" gets the specification "large class of", as the only optimization problem the authors are solving is Equation (5), while "large class" strengthens the wrong semantics that it concerns a (not yet explained) other class of optimization problems.  I guess that "large class" does not refer to optimization problems but to a large class of probability distributions P_\nu, large classes of privatization schemes Q, etc.* Line 124: Next, the reader stumbles upon "the differential privacy area, where the basic mechanisms have been few".  What is a basic mechanism?  It could refer to privatization mechanisms, but why are they called basic (rather than privatization) then?  A much simpler and less confusing formulation would be "... to the differential privacy area where allmost no results are known on the optimality of privatization mechanisms." (if at least this is what the authors mean).* Somehow, the reader also needs to realize that the "extremal" from the title is a synonym for the "optimal" in this paragraph.  Avoiding the use of synonyms often avoids confusion.* Line 125 mentions a linear program over "the alphabet |\mathcal{X}|.  Probably the "alphabet size" is meant (|.| gives a number).  By the way, in line 64, \mathcal{X} was not introduced (and the reader learns here that it is called "alphabet").  Up to here, the reader may think that \mathcal{X} is an uncountably infinite set (e.g. vector space).  This impression is strengthened by the usage of integrals in the formulas up to here.* Line 126-128 seems to revoke the original stronger claim that optimal mechanisms can be found using this paper.  It now seems to hard to compute, and the paper will only give optimal mechanisms in certain "regimes", which usually means something like "the optimality result holds for the extremes (e.g. 0 and infinity), and the more you are in the middle between these extremes, the large is the suboptimality.".   Still, later in lines 144-146, the text compares with related work and says that this other work only provides "near-optimality" while this paper does better.* line 129 announces bounds on differential privacy: Suddenly, it is not local anymore. What exactly is lower bounded?  In terms of what is the lower bound given?  The statement is quite vague.* In "recovering some known results.", what is "recover"?  It sounds as this "recovering" is not giving new results, but repeating what has been published earlier?  Or do you mean that you adapt existing bounds to a new setting where they originally did not yet apply?* In Line 129, The sentence "As a motivating example, we provide optimal mechanisms" is again confusing.  I thought that already line 123 announced optimal mechanisms?  So are the optimal mechanisms a motivating example for the motivating mechanisms?  What is the difference?* Ine line 134, it is unclear what "a converse to the optimization problem" means, the following phrase (and show ...) does not disambiguate this completely.In conclusion, even though the authors indeed provide in the mentioned lines a summary of their contributions, it is vague, and the text provides many opportunities to the average reader (who does not have author responses and fellow reviewers) for getting confused.  In allmost all cases, simple reformulations exist which make the text clearer, more precise, less ambiguous and less confusing.W.r.t. Comment 6:  Authors say: "The specific quoted phrase is embedded in a larger sentence. Reading the entire sentence, local privacy DOES NOT mean that individuals reveal their personal information, but instead local privacy is required IN THE CONTEXT where individuals reveal their personal information. Local differential privacy is when users first privatize their personal data (X_i's) and then release privatized views (Y_i's) to a data collector. Therefore, the data collector never observes the X_i's but instead has access to the Y_i's." It indeed often happens that the authors write sentences which have certain grammatical features which do not allow an interpretation consistent with other sentences or parts of sentences.  In this case, the quoted phrase IS NOT embedded in a larger sentence which may let the reader suspect he is being misled.  In particular, the complete sentence is "The need for data privacy appears in two different contexts: one in the data collection scenario, as in when individuals disclose their personal information (e.g. voluntarily on social network sites).".  Later, the paper says that this first scenario is local differential privacy.  It is true that there are other sentences in the paper which describe local differential privacy better, but why then first give a wrong description?My specific comment refers amongst others to the fact that the example is clearly not a good one.  Users providing their information on a social network site do not privatize their information in some complicated way.  They do not know the (one or more) data analysts who will analyze the data, and hence even if they would apply a privatization scheme, they would not be able to use the content in this paper to select an optimal privatization scheme.  Even if we ignore the specific ill-chosen example, it can't be denied that the paper assumes that there is only one data analyst willing to test only one hypothesis.  In practice, this is not realistic.  Data is published, potentially in some privatized way, but then usually a data analyst will have more then a binary question.Authors say: "7A) Local differential privacy has been studied in the literature. Our notations, definitions, and problem statement have been chosen to be consistent with recent prior papers on the subject (which we cite in our paper). Specifically, the terms "privatized views", "privatized outputs", and "privatization mechanism" are chosen to be consistent with their use in [10]."Ideally, a paper is self-contained as should not require from the reader that he digs into a lot of other literature to grasp the main message.Even if the authors choose to not define terms already used in the literature, why then let the reader guess for a long time and not simply start by saying that the concepts and notations in this paper are defined formally in [10] ?More generally, several boosting sentences such as "We first give a formal definition ..." let the reader expect a certain level of rigorousness, and cause the reader to think his missed something in this paper while he is not able to find it.  If you don't plan to give a formal definition, or if you give it in supplementary materials, ... my advice is to simply tell this to the reader.Still, it is my belief (now understanding things better), that there are simple ways to convey most of this essential information to the reader without using significantly more space.  Even if a clearer explanation would require more space, I feel it is worth moving one or two technical derivations to the supplementary materials.Authors say: "7B) Binary hypothesis testing is a canonical statistical inference problem with a wide range of applications. As is evident from [10] and this paper, understanding the fundamental limits to private binary hypothesis testing is a challenging problem. The more general problem of private M-ary hypothesis testing is also interesting, but is outside of the scope of this paper."I agree with that.  But then, this is an interesting point of discussion and possible future work.  Why not have a short "conclusion and future directions" section where you outline the limitations of the current paper and remaining open questions?w.r.t. comment 16, I feel that examples, discussions and illustrations don't necessarily require figures.    Well-written sentences often can help a lot.