Sparse PCA with Oracle Property
In this paper, we study the estimation of the k-dimensional sparse principal subspace of covariance matrix Σ in the high-dimensional setting. We aim to recover the oracle principal subspace solution, i.e., the principal subspace estimator obtained assuming the true support is known a priori. To this end, we propose a family of estimators based on the semidefinite relaxation of sparse PCA with novel regularizations. In particular, under a weak assumption on the magnitude of the population projection matrix, one estimator within this family exactly recovers the true support with high probability, has exact rank-k, and attains a s/n statistical rate of convergence with s being the subspace sparsity level and n the sample size. Compared to existing support recovery results for sparse PCA, our approach does not hinge on the spiked covariance model or the limited correlation condition. As a complement to the first estimator that enjoys the oracle property, we prove that, another estimator within the family achieves a sharper statistical rate of convergence than the standard semidefinite relaxation of sparse PCA, even when the previous assumption on the magnitude of the projection matrix is violated. We validate the theoretical results by numerical experiments on synthetic datasets.
The abstract is an accurate summary of the paper. The paper proposes two new estimators for a sparse principal component subspace. The estimators are based on a novel regularization of a know semidefinite programming relaxation of the  rank-constrained principal subspace problem. Consistency-like properties of the estimators are shown with faster rates that previously known (nearly optimal in some regimes). Suitable optimization algorithms are described and tested on synthetic experiments, showing improved accuracy over previous methods. The new estimators also have weaker structural assumptions on the covariance matrix to guarantee convergence. The asymptotic distribution of one of the estimators is derived, making it possible to construct confidence regions.The results are fairly interesting, the presentation is clear, the results seem quite original. The significance follows from the improved theoretical rates shown as well as the improved performance in synthetic experiments.Specific comments:82: optimizaiton83: optima -> optimum141: Confusing notation: the empirical covariance matrix is S as well as \hat \Sigma. Later, S denotes the support.213: What is L?220: "If" sentence is incomplete.