Query Complexity of Bayesian Private Learning
We study the query complexity of Bayesian Private Learning: a learner wishes to locate a random target within an interval by submitting queries, in the presence of an adversary who observes all of her queries but not the responses. How many queries are necessary and sufficient in order for the learner to accurately estimate the target, while simultaneously concealing the target from the adversary? Our main result is a query complexity lower bound that is tight up to the first order. We show that if the learner wants to estimate the target within an error of ✏, while ensuring that no adversary estimator can achieve a constant additive error with probability greater than 1/L, then the query complexity is on the order of L log(1/✏) as ✏ ! 0. Our result demonstrates that increased privacy, as captured by L, comes at the expense of a multiplicative increase in query complexity. The proof builds on Fano's inequality and properties of certain proportional-sampling estimators.
- Summary & Strength  The paper studies a sequential learning with binary feedback with a privacy concern where the learning objective is to locate a latent value with some confidence while hiding it from a spying adversary who observes all the queries but not the responses. The authors obtain a tight fundamental limit of the query complexity using Fano’s inequality, which is a standard tool for the lower bound analysis. The fundamental limit matches the query complexity of an upper bound algorithm (Replicated Bisection strategy) in the asymptotic order which is originally proposed in [11, 15]. The query complexity analysis shows a privacy-efficiency trade-off. The proof seems sound, and the paper is easy to follow.   - Weakness & Comment  The proposed model is somewhat artificial. In particular, all the responses are assumed to be correct, whereas noisy response is common in practice. Hence, it limits to connect the considering problem to a motivating example in practice. In addition, there is a line of works e.g., [BA 2008, 13], that study the query complexity even under noisy responses (which lead to interesting applications) although there is no privacy concern. However, there is no justification or motivation on the noiseless response.   - Minor comments  I’m not sure that the notion of the random seed Y is necessary.  Line 34: identity function > indicator function Line 44: $R^i := \emptyset$ > $R^1 := \emptyset$ Line 46: “guess’ > “guess” Line 61: any an adversary > any adversary Line 163: $L \log (1/\epsilon)- L (\log L -1)$ > $L \log (1/\epsilon)- L (\log L -1) - 1$ Eq. 5.7: ${\cal E}_{i, j}$ is used without defining it.  -- [BA 2008]: Ben-Or, Michael, and Avinatan Hassidim. "The bayesian learner is optimal for noisy binary search (and pretty good for quantum as well)." Foundations of Computer Science, 2008. FOCS'08. IEEE 49th Annual IEEE Symposium on. IEEE, 2008.