PCA as a Cure for the Curse of Dimensionality I would like some clarification as to how principal component analysis mitigates the Curse of Dimensionality problem. My particular interest is in curbing overfitting in my modelling, or more specifically parameter count. If I use all 30 of my features I will have a model with 30 parameters: this is too large a number for my sample size, overfit is almost guaranteed. I am told that I should rather build my model with the first 3 principal components of my feature set and thus have only a 3 parameter model, and apparently mitigate my overfitting problem. But then I have computed 30x30 elements for my eigenvector matrix and 3 parameters for my model, I have fitted 900+3 parameters to the data. Now I have gone from a model with a maximum parameter count of 30 to a model with 903 parameters. How have I evaded the Curse of Dimensionality? It is really not obvious to me. An additional issue is the high variance of the elements of the eigenvector matrix, I have noted that relatively small changes to the feature data cause considerable variation in these elements, sometimes even changing signs. They are more unstable than the parameters of the model that I am trying to fit.


But then I have computed 30x30 elements for my eigenvector matrix and 3 parameters for my model, I have fitted 900+3 parameters to the data.

The possible solutions for the parameters relating to the features are strongly limited. You are effectively only fitting 3 parameters. Because the potential solutions $\hat{Y} = \beta_1 X_1 + \beta_2 X_2 + \dots + \beta_{30} X_{30}$  lie in a 3d space.
