In the problem of best linear predictor, why is $E(XX')$ positive definite equivalent to $E(XX')$ being invertible? I came across the following statement in a textbook when discussing the classic best linear predictor problem in statistics. It says $E[XX']$ being positive definite is equivalent to it being invertible. I'm wondering why it is the case here because in general an invertible matrix is not necessarily positive definite. (Positive definiteness and nonsingularity makes no difference in the calculation of the best linear predictor coefficient $E[XX']^{-1}E[XY]$, but it does make a difference for arguing that the best linear predictor coefficient achieves the global minimum of the objective function.) Thanks!


This is because $E[XX']$ is inherently positive semi-definite, as for any non-random $z \in \mathbb{R}^n$:
\begin{align}
z'E[XX']z = E[z'XX'z] = E[(X'z)'(X'z)] \geq 0. 
\end{align}
Now the result follows from the fact that a positive semi-definite matrix that is invertible is positive definite.
