Statistical Active Learning Algorithms
We describe a framework for designing efficient active learning algorithms that are tolerant to random classification noise and differentially-private. The framework is based on active learning algorithms that are statistical in the sense that they rely on estimates of expectations of functions of filtered random examples. It builds on the powerful statistical query framework of Kearns [30]. We show that any efficient active statistical learning algorithm can be automatically converted to an efficient active learning algorithm which is tolerant to random classification noise as well as other forms of "uncorrelated" noise. We show that commonly studied concept classes including thresholds, rectangles, and linear separators can be efficiently actively learned in our framework. These results combined with our generic conversion lead to the first computationally-efficient algorithms for actively learning some of these concept classes in the presence of random classification noise that provide exponential improvement in the dependence on the error over their passive counterparts. In addition, we show that our algorithms can be automatically converted to efficient active differentially-private algorithms. This leads to the first differentially-private active learning algorithms with exponential label savings over the passive case.
The authors propose a class of active learning 
      algorithms, termed 'active statistical' algorithms that is supposed to 
      combine the noise robustness, and differential privacy of the standard 
      statistical query (SQ) framework, with the active learning setting in 
      which one wishes to minimizes the number of labels one requires by 
      actively (adaptively) selecting which data points to label. As 
      defined, the active SQ algorithms asks queries that consist of two parts: 
      a 'filter' f:X->[0,1] representing the probability that a point X 
      should be included in the current query, and the function g:X x 
      {-1,1}->[-1,1], which is a function of a point-label pair. 
      Additionally, as part of the query, there are two tolerance parameters, 
      t1,t2, and the guarantee is that provided the measure of examples defined 
      by f is at least t1, one receives a noisy estimate of E[g(x,y)} to within 
      t2, where the expectation is defined over the measure induced by f. 
      In general, I doubt this new model will be as helpful, or have as 
      much mathematical structure as the general SQ model. That said, it does 
      seem to be `the right' model for capturing noise tolerance in an active 
      learning setting (which yields differential privacy), and the algorithmic 
      results, while not earth-shattering, provide a nice illustration of the 
      power of this model. It would be nice to include a little more 
      discussion of the relation between the general SQ framework and the 
      proposed model--specifically, the fact that the proposed model of queries 
      can be implemented in the SQ framework, just with a tolerance that scales 
      according to the measure of the portion of the distribution in 
      consideration for each query.