Fast Classification Rates for High-dimensional Gaussian Generative Models
We consider the problem of binary classification when the covariates conditioned on the each of the response values follow multivariate Gaussian distributions. We focus on the setting where the covariance matrices for the two conditional distributions are the same. The corresponding generative model classifier, derived via the Bayes rule, also called Linear Discriminant Analysis, has been shown to behave poorly in high-dimensional settings. We present a novel analysis of the classification error of any linear discriminant approach given conditional Gaussian models. This allows us to compare the generative model classifier, other recently proposed discriminative approaches that directly learn the discriminant function, and then finally logistic regression which is another classical discriminative model classifier. As we show, under a natural sparsity assumption, and letting s denote the sparsity of the Bayes classifier, p the number of covariates, and n the number of samples, the simple ( 1 -regularized) logistic regression classifier achieves the fast misclassification error rates of O s log p n , which is much better than the other approaches, which are either inconsistent under high-dimensional settings, or achieve a slower rate of O s log p n .
I have read the paper ``fast classification rates for high-dimensional conditional Gaussian models".The paper studiesthe problem of binary classificationusing a Gaussian model and provides some theoretical results on the convergence of the classification error rates (compared to the Bayes classifier). The paper presents some nice theoretical results and is interesting to some extent. I am generally positive about the paper but I have the following concerns. First, it is about the practical relevance.One of the very important point about classification is that we need a tuning free trained classifier; when a classifierhas tuning parameters that need to be tuned, we usually use data splittingto decide such tuning parameters. Therefore, it is more relevant to either studythe error rates of a tuning free classifier, or that of a classifier with tuning parametersselected by data splitting.  This is important for that, while the convergence rate of the "ideal tuning parameters"may be fast, the convergence rate of the classifier that is actually used could be very slow, simply because of the convergence of the data-driven tuning parameters to the ideal tuning parameters are slow.  In terms of this, the paper needs to (a) compare more carefully with recent tuning-free classifiers, (b) address the effect of selecting the tuning parameters by data-splitting. Second of all, the paper has put all the focus on the convergence rate,does notcontain any insightful comparison of the computing speed.The penalization methodsthe authors present are all computationally slow when $p$ is large. It is desirable tohave a detailed comparison about the computational complexity or speed. Third of all, given this is a well-studied topics, it is desirable to comparewith different methods using at least 1-2 real data sets (say, microarray data sets). The methods that need to compare include: those discussed in the paper [3,7,13,15,17] if the method is easy to implement as well as support vector machine (SVM) and random Forest. Fourth, the paper comments that assuming $\Sigma^{-1} (\mu_1 - \mu_0)$ is sparse is weaker than assuming both $\Sigma^{-1}$ and $(\mu_1 - \mu_0)$ are sparse. Mathematically, this is true. However, the more relevant point is that, whether we have an important application example(say, in genomics or genetics) where $\Sigma^{-1} (\mu_1 - \mu_0)$ is sparse but either $\Sigma^{-1}$ is non-sparse or$(\mu_1 - \mu_0)$ is non-sparse. If we don't have such application examples in areas many people care, then arguing how two assumptions are different from each other is largely due to mathematicalcuriosity, not due to scientific interest. Last, the presentation can be improved. The material can be streamlined so that the main contribution can be more highlighted.