Answers.Body: <p>Because it selects both <code>Xtrain</code> and <code>Xtest</code> from the space of two selected principal components. Hence, the 90% accuracy is in that 2-D selected space. </p>
  
  <p>This fact that the ratio in PCA stands the information, depends on the distribution of the data and it's not true at all. </p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-01-05T10:36:46.887Z
Answers.LastActivityDate: 2019-01-05T10:43:41.540Z
Answers.LastEditDate: 2019-01-05T10:43:41.540Z
Answers.UserDisplayName: OmG
Body: <p>I'm learning machine learning by looking through other people's kernel on Kaggle, specifically this <a href="https://www.kaggle.com/raghuchaudhary/mushroom-classification/data" rel="nofollow noreferrer">Mushroom Classification kernel</a>.</p>
  <p>The author first applied PCA to the transformed indicator matrix. He only used 2 principal components for visualization later. Then I checked how much variance it has maintained, and found out that only 16% variance is maintained.</p>
  <pre><code>in [18]: pca.explained_variance_ratio_.cumsum()
  out[18]: array([0.09412961, 0.16600686])
  </code></pre>
  <p>But the test result with 90% accuracy suggests it works well.</p>
  <p>If variance stands for information, then how can the ML model work well when so much information is lost?</p>
  
Tags: <machine-learning><principal-component-analysis>
Title: Why does PCA work well while the total variance retained is small?
UserDisplayName: Bicheng
ThreadText: # Why does PCA work well while the total variance retained is small?
  
  (Asked by: username_0 on 2019-01-05)
  
  I'm learning machine learning by looking through other people's kernel on Kaggle, specifically this [Mushroom Classification kernel](https://www.kaggle.com/raghuchaudhary/mushroom-classification/data).
  
  The author first applied PCA to the transformed indicator matrix. He only used 2 principal components for visualization later. Then I checked how much variance it has maintained, and found out that only 16% variance is maintained.
  
  ```
  in [18]: pca.explained_variance_ratio_.cumsum()
  out[18]: array([0.09412961, 0.16600686])
  
  ```
  
  But the test result with 90% accuracy suggests it works well.
  
  If variance stands for information, then how can the ML model work well when so much information is lost?
  
  **Tags:** machine-learning, principal-component-analysis
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-01-05. Score: 0
  
  Because it selects both `Xtrain` and `Xtest` from the space of two selected principal components. Hence, the 90% accuracy is in that 2-D selected space.
  
  This fact that the ratio in PCA stands the information, depends on the distribution of the data and it's not true at all.
  
  
  ---
  
  