Adaptivity to Local Smoothness and Dimension in Kernel Regression
We present the first result for kernel regression where the procedure adapts locally at a point x to both the unknown local dimension of the metric space X and the unknown Hölder-continuity of the regression function at x. The result holds with high probability simultaneously at all points x in a general metric space X of unknown structure.
This paper considers the noisy regression setting with 
      n data (X_i,Y_i) such that \E(Y | X=x) =f(x). The underlying function f is 
      locally Holder smooth with associated coefficient \alpha_x. The 
      distribution \mu underlying X is locally d_x dimensional. The paper 
      proposes a method to adapt locally the bandwidth in kernel regression to 
      the unknown \alpha_x and d_x. The authors propose an adaptive estimate 
      \hat f of f that is such that, if n large enough, with probability 1 - 
      \delta, for any x |\hat f(x) - f| < C 
      (\log(n/\delta)/n)^{\alpha_x/(2\alpha_x + d)}. They assume however 
      that: (i) An upper bound on \sup_x d_x is available (ii) For each 
      x, the dimension d_x is "attained" provided that we consider a small 
      enough neighbourhood This paper is a combination of results in [4] 
      and [5] (see bibliography in the paper), i.e. local Lepski's method, and 
      [9], i.e. adaptation to the intrinsic dimension: the results are 
      interesting but not surprising in view of these works. I did not read the 
      proofs in full details, but what I checked is correct and the results are 
      plausible. I have some questions: - Results concerning adaptive 
      (to the smoothness) methods such as Lepski's method are in general proved 
      uniformly on a class of function (The alpha Holder functions in your 
      case). Could you do the same? - What about confidence intervals for 
      your adaptive estimate? - You assume (ii) for the intrinsic dimension. 
      Do you really need this? It is not necessary in Lepski's method to assume 
      that the smoothness \alpha is available: adaptive methods will tend to 
      over-smooth, but this is not a problem. Could you have something similar 
      with the intrinsic dimension? To me, the main problem with this 
      paper is the writing, which is not precise and not intuitive. Here is an 
      un-exhaustive list of the issues I spotted: -p1l43: the minimax 
      optimal oracle rate is indeed n^{-2\alpha/(2\alpha+d)}, but you should 
      definitely mention the minimax optimal adaptive rate, which is more 
      relevant to your problem, i.e. \min_{\hat f} \max_{\alpha} \max_{f 
      \alpha Holder} \E \|\hat f - f\|_{\infty} 
      (n/\log(n))^{2\alpha/(2\alpha+d)} > C where C is a constant. In 
      other words, the minimax optimal oracle rate is n^{-2\alpha/(2\alpha+d)}, 
      but the minimax optimal rate is (n/\log(n))^{2\alpha/(2\alpha+d)} 
      (inevitable additional (\log(n))^{2\alpha/(2\alpha+d)} to pay for the 
      non-knowledge of \alpha). Also, it is the rate you obtain in your result. 
      -p2l64 and 76: Are \rho and \|.\| the same connected/ So \rho does 
      actually define a norm? -p2l90: What is B(x,r)? I assumed it is the 
      ball of centre x and radius r but you should state it. -p2l98: What is 
      w.l.o.g.? Without loss of generality? -p3l150: the constant N is very 
      important. Could you give it explicitly? -p3l161 (main equation): You 
      have C_{n,\delta} = C\log(n/\delta): there is a dependency in 
      \log(n)^{2\alpha/(2\alpha+d)} in your bound. Please make it explicit. This 
      is anyway matching the lower (non-oracle) bound. -p5l228: Please 
      provide a proof for Lemma 2: shouldn't an additional 
      log(1/(\epsilon\delta)) appear in the bound due to union bound on 
      S_{\epsilon}? -p6l286: citation for McDiarmid inequ please 
      -p7l153: The proof of Thm 1 is to me correct but redacted in a non 
      precise way. Instead of saying "Assume that the statement of Lemma ..." is 
      verified, it would be more clear to precise exactly on which event the 
      Theorem holds. -p8l395: Also the proof seems un-necessary complicated: 
      you could maybe skip the definition of \bar h and prove that: (1) if 
      c\tilde h < \hat h, then |\hat f_{n, \hat h}(x) - f(x)|< \hat 
      \sigma_{\tilde h} < C \tilde R(\tilde h) and (2) the probability that 
      c\tilde h > \hat h is very small since it is not likely that |\hat 
      f_{n, c\tilde h} - \hat f_{n, \hat h}| > \hat \sigma_{\hat h} for such 
      a \hat h. Unify the notation log and ln. Is your log a log in base 
      2? And also it would be great if you could extend the discussion 
      on your results, and the intuitive explanations. The paper is to my mind 
      quite hard to read, because very technical (sometimes without necessity). 
      Also you should add some references on Lepski's method: the upper and 
      lower bounds for this method were not provided in [4], [5] but 
      before.