Answers.Body: <p>PCA works well where data sample space is linear. If data sample space is not linear or it is manifold data then model without PCA may perform better than model using PCA.</p>
  
  <p><a href="https://i.sstatic.net/FtQsj.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/FtQsj.png" alt="Manifold Data"></a></p>
  
  <p>In the given image you can see, data is manifold. In this type of data, PCA, which is based on projection technique does not work well. That's why we use manifold learning technique to handle these cases.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-05-01T11:46:12.923Z
Answers.LastActivityDate: 2020-05-01T11:46:12.923Z
Answers.UserDisplayName: VIJAY
Answers.Body: <p>PCA can make models worse,  imagine data points scattered along two elongated parallel rectangles. The axis with the greatest variation will be parallel to the rectangles but doesn't provide any benefit in classifying the points.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-05-01T05:19:52.727Z
Answers.LastActivityDate: 2020-05-01T05:19:52.727Z
Answers.UserDisplayName: FourierFlux
Body: <p>In case I had a prediction model and decided to add a PCA step prior to the model, is it theoretically possible/impossible that the number of output dimensions that is better for all tests may perform worse than the model without PCA?</p>
  <p>My question comes from the fact that I want to add a PCA step prior to a model and hyperparameterize the PCA output dimension from 1 to N (N being the number of dimensions in the original dataset) and I wanted to know if there is any theoretical basis that there is no case in which performing this previous step could have a worse performance than the previous model.</p>
  <p>Especially, my doubt comes if the best PCA case from a selection of dimensions from 1-N is always better than the best case without PCA.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-04-30T15:44:44.130Z
Comments.Text: What do you mean by "number of output dimensions that is better". How is an "output dimension" better than another? Please, edit your post to clarify this!
Comments.UserDisplayName: nbro
Tags: <machine-learning><hyperparameter-optimization><principal-component-analysis><dimensionality-reduction>
Title: Is it theoretically possible (or impossible) that principal component analysis worsens the performance of the model?
UserDisplayName: Angelo
ThreadText: # Is it theoretically possible (or impossible) that principal component analysis worsens the performance of the model?
  
  (Asked by: username_0 on 2020-04-30)
  
  In case I had a prediction model and decided to add a PCA step prior to the model, is it theoretically possible/impossible that the number of output dimensions that is better for all tests may perform worse than the model without PCA?
  
  My question comes from the fact that I want to add a PCA step prior to a model and hyperparameterize the PCA output dimension from 1 to N (N being the number of dimensions in the original dataset) and I wanted to know if there is any theoretical basis that there is no case in which performing this previous step could have a worse performance than the previous model.
  
  Especially, my doubt comes if the best PCA case from a selection of dimensions from 1-N is always better than the best case without PCA.
  
  **Tags:** machine-learning, hyperparameter-optimization, principal-component-analysis, dimensionality-reduction
  
  > **Comments:**
  > **username_1**: What do you mean by "number of output dimensions that is better". How is an "output dimension" better than another? Please, edit your post to clarify this!  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2020-05-01. Score: 1 | Accepted answer
  
  PCA works well where data sample space is linear. If data sample space is not linear or it is manifold data then model without PCA may perform better than model using PCA.
  
  [![Manifold Data](https://i.sstatic.net/FtQsj.png)](https://i.sstatic.net/FtQsj.png)
  
  In the given image you can see, data is manifold. In this type of data, PCA, which is based on projection technique does not work well. That's why we use manifold learning technique to handle these cases.
  
  
  ---
  
  ### Answer by username_3 on 2020-05-01. Score: 1
  
  PCA can make models worse, imagine data points scattered along two elongated parallel rectangles. The axis with the greatest variation will be parallel to the rectangles but doesn't provide any benefit in classifying the points.
  
  
  ---
  
  