Is this description of linear discriminant analysis (LDA) correct? Hang Nguyen writes the following in "Machine learning basics (part 14): Linear Discriminant Analysis":

If there are two classes then the LDA draws one hyperplane and projects the data onto this hyperplane in such a way as to maximize the separation of the two categories. This hyperplane is created according to the two criteria considered simultaneously:

*

*Maximizing the distance between the means of two classes;

*Minimizing the variation between each category.


Is that correct?

Fisher's approach was ingenious and intuitively clear: separate the populations $\Pi_i$ based on the linear function $\mathbf a^\top \mathbf x$ that maximizes the ratio of between-groups of sum of squares to within-group sum of squares:
Consider $\mathbf y=\mathbf X\mathbf a, ~\mathbf X$ being the data matrix. Total sum of squares $\mathbf y^\top \mathbf H\mathbf y=\mathbf a^\top \mathbf X^\top\mathbf H\mathbf X\mathbf a$ can be decomposed into sum of within-groups sum of squares $\sum\mathbf y_i^\top\mathbf H_i\mathbf y_i=\sum \mathbf a^\top \mathbf X_i^\top\mathbf H_i\mathbf X_i\mathbf  a:= \mathbf a^\top \mathbf W\mathbf a, $ (here $\mathbf H_i$ being the centering matrices) and  between-groups sum of squares $\sum_i n_i(\bar y_i-\bar y) :=\mathbf a^\top\mathbf B\mathbf a. $
We seek to find $\mathbf a$ that maximizes the ratio (as remarked by statmerkur) $$ \frac{\mathbf a^\top\mathbf B\mathbf a}{\mathbf a^\top\mathbf W\mathbf a}$$ which would be nothing but the largest eigenvalue's eigenvector of $\mathbf W^{-1}\mathbf B. $

Coming to the stated blog: while I won't say it is outright wrong, it is indeed problematic:
$\bullet$ The author defined $\mathbf S_b$ in terms of $\boldsymbol\mu_i,$ the population means. But then used sample within groups matrix $\sum_i (n_i-1) \mathbf S_i=\sum_i\sum_{j=1}^{n_i}(\mathbf x_{ij} -\bar{\mathbf x}_i) (\mathbf x_{ij} -\bar{\mathbf x}_i)^\top.$  The former is used when the population parameters $(\boldsymbol\mu_i, \mathbf\Sigma) $ are known. Otherwise, in place of $\mathbf S_b, ~\sum_i(\mathbf{\bar x}_i-\mathbf{\bar x}) (\mathbf{\bar x}_i-\mathbf{\bar x}) ^\top$ has to be substituted. The author is plausibly confusing things using Fisher's sample LDs.
$\bullet$

Maximizing the distance between the means of two classes;

Now they are talking about two classes instead of $g$ classes. For $g=2, $ the interpretation would be to find $\mathbf a$ such that maximum separation is achieved between $\bar y_1$ and $\bar y_2, $ measured in standard deviation units.

References:
$\rm [I]$ Applied Multivariate Statistical Analysis, Wolfgang Karl Härdle, Léopold Simar, Springer-Verlag, $2015, $ sec. $14.2, $ pp. $418-419.$
$\rm [II]$ Applied Multivariate Statistical Analysis, Richard A. Johnson, Dean A. Wichern, Pearson, $2013, $ sec. $11.6, $ pp. $622-623; ~590.$
