Kernel Feature Selection via Conditional Covariance Minimization
We propose a method for feature selection that employs kernel-based measures of independence to find a subset of covariates that is maximally predictive of the response. Building on past work in kernel dimension reduction, we show how to perform feature selection via a constrained optimization problem involving the trace of the conditional covariance operator. We prove various consistency results for this procedure, and also demonstrate that our method compares favorably with other state-of-the-art algorithms on a variety of synthetic and real data sets.
In this paper, authors propose a new nonlinear feature selection based on kernels. More specifically, the conditional covariance operator has been employed to measure the conditional independence between Y and X given the subset of X. Then, the feature selection can be done by searching a set of features that minimizing the conditional independence. This optimization problem results in minimizing over matrix inverse and it is hard to optimize it. Thus, a novel approach to deal with the matrix inverse problem is also proposed. Finally, the consistency of the feature selection algorithm is proved. Through experiments, the authors showed that the proposed algorithm compares favorably with existing algorithms.
The paper is clearly written and the proposed algorithm is technically sound. Overall, I like the algorithm.
Detailed comments:
1. In the proposed algorithm, some information about tuning parameters are missing. So, please include those information in the paper. For example, lambda_2 in Eq. (20). Also, how to initialize the original w?
2. It would be great to compare with other kernel based feature selection algorithms such as HSIC-LASSO and KNIFE. Also, related to the first comments, is it possible to further improve the proposed algorithm by properly initializing the w? For instance, HSIC-LASSO is a convex approach and can have reasonable features. 
3. The results of TOX-171 is pretty interesting. It is nice to further investigate the reason why the proposed approach outperforms rest of it.
4. It is nice to have comparison of redundancy rate. In feature selection community tends to use this measure for evaluation. 
Zhao, Z., Wang, L., and Li, H. (2010). Efficient spectral feature se lection with minimum redundancy. In AAAI Conference on Artificial Intelligence (AAAI) , pages 673–678.