Regression-tree Tuning in a Streaming Setting
We consider the problem of maintaining the data-structures of a partition-based regression procedure in a setting where the training data arrives sequentially over time. We prove that it is possible to maintain such a structure in time O (log n) at any time step n while achieving a nearly-optimal regression rate ofÕ n −2/(2+d) in terms of the unknown metric dimension d. Finally we prove a new regression lower-bound which is independent of a given data size, and hence is more appropriate for the streaming setting.
The authors derive a new procedure to estimate a 
      recursive-partition prediction rule in the streaming framework. 
      Theoretical analyses demonstrate that the procedure is computationally 
      efficient and attains the minimax prediction error rate, up to a log 
      factor. A small empirical analysis is in agreement with the theory. 
      The paper is excellent: the authors have produced an intuitive 
      streaming algorithm with nearly sharp theoretical guarantees in terms 
      of intrinsic dimension. The exposition is very clear and easy to 
      follow. I have been hoping for some time to see a streaming regression 
      tree paper like this. Properly publicized, this work is liable to have 
      a big impact in the large-scale learning community. Regarding 
      originality, the authors have overlooked some work on this problem -- 
      but that work focuses on bounding the difference between the streaming 
      tree and the corresponding batch tree, which is a different question 
      altogether. So there's no harm done. I've listed some references 
      below. I warmly recommend acceptance. Minor comments. 
      p.2: "everywhere on the space rho" -> "everywhere on the space 
      calX" p.2: "The metric (calX, rho)" -> "The metric space (calX, 
      rho)" p.3: "an new guess" -> "a new guess" p.3: "quantifieable" 
      -> "quantifiable" [***** SPELL CHECK *****] p.3: "there exist C" 
      -> "there exists C" References. @inproceedings{, 
      author = {Domingos, Pedro and Hulten, Geoff}, booktitle = 
      {Proceedings of the 6th ACM SIGKDD International Conference on Knowledge 
      Discovery and Data Mining}, title = {Mining high-speed data streams}, 
      pages = {71--80}, year = {2000} } @inproceedings{, 
      author = {Hulten, Geoff and Spencer, Laurie and Domingos, Pedro}, 
      booktitle = {Proceedings of the 7th ACM SIGKDD International 
      Conference on Knowledge Discovery and Data Mining}, title = {Mining 
      time-changing data streams}, pages = {97--106}, year = {2001} 
      } @inproceedings{, author = {Pfahringer, Bernhard and 
      Holmes, Geoffrey and Kirkby, Richard}, booktitle = {Advances in 
      Knowledge Discovery and Data Mining: Proceedings of the 12th Pacific-Asia 
      Conference (PAKDD)}, title = {Handling numeric attributes in Hoeffding 
      trees}, volume = {5012}, pages = {296--307}, publisher = 
      {Springer}, year = {2008} } @article{, author = 
      {Ben-Haim, Yael and Tom-Tov, Elad}, title = {A streaming parallel 
      decision tree algorithm}, journal = {Journal of Machine Learning 
      Research}, volume = {11}, pages = {849--872}, year = {2010} 
      } NB: I have read the author rebuttal.