Greedy Feature Construction
We present an effective method for supervised feature construction. The main goal of the approach is to construct a feature representation for which a set of linear hypotheses is of sufficient capacity -large enough to contain a satisfactory solution to the considered problem and small enough to allow good generalization from a small number of training examples. We achieve this goal with a greedy procedure that constructs features by empirically fitting squared error residuals. The proposed constructive procedure is consistent and can output a rich set of features. The effectiveness of the approach is evaluated empirically by fitting a linear ridge regression model in the constructed feature space and our empirical results indicate a superior performance of our approach over competing methods.
This paper presents an effective method for feature construction by exploiting the scalability of existing algorithms for training linear models while overcoming their low capacity on input features. The authors first embed the data into a space with a high capacity set of linear hypotheses, and then use a greedy approach to construct features by fitting residuals.  Overall, I think the contribution of this paper is significant. As noted by the authors, linear models are frequently used for learning on large scale data sets. Yet, the set of linear hypotheses defined on input features is usually of low capacity, thus leading to inaccurate descriptions of target concepts. By embedding the input data into a high capacity set of linear hypotheses and then constructing features in a greedy manner, the proposed method can construct a high capacity feature space. The experimental results also demonstrate the effectiveness of the proposed approach.  
This paper presents an effective method for feature construction by exploiting the scalability of existing algorithms for training linear models while overcoming their low capacity on input features. The authors first embed the data into a space with a high capacity set of linear hypotheses, and then use a greedy approach to construct features by fitting residuals.  Overall, I think the contribution of this paper is significant. As noted by the authors, linear models are frequently used for learning on large scale data sets. Yet, the set of linear hypotheses defined on input features is usually of low capacity, thus leading to inaccurate descriptions of target concepts. By embedding the input data into a high capacity set of linear hypotheses and then constructing features in a greedy manner, the proposed method can construct a high capacity feature space. The experimental results also demonstrate the effectiveness of the proposed approach.   Despite of its significance, I think the paper can be improved in the following aspects.  First, the flow of this paper should be improved. Although English grammar is ok, this paper is not very easy to follow. I understand that the authors need to meet the page limits, but it would be better if the authors can include several sentences in the introduction part to summarize the proposed method, which serve as a clue for the following paragraphs.  Second, the effectiveness of the proposed method has been clearly demonstrated by the experiments. Is it possible for the authors to evaluate the classification accuracy for the proposed approach?    Therefore, my opinion for this paper is borderline.