Body: <p>I am working on a hybrid CNN-SVM classification using python. I tried to get the final output features of the CNN model(flatten layer) to be fed to the SVM classifier. The output shape obtained from the flatten layer was (600,32786).For computational reasons, I wanted to reduce the shape of the obtained features to (600,250). I was looking for a way to do that without losing relevant features, and I read about the PCA technique.
  I saved the output features in a(.npy) file. I tried the following:</p>
  <pre><code>X= np.load('features.npy')
  pca_250 = PCA(n_components=250)
  pca_250.fit(test_images)
  Y = pca_250.transform(test_images)
  </code></pre>
  <p>Is this correct?</p>
  <p>Regards</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-09-17T15:59:08.403Z
Comments.Text: You’re basically guaranteed to lose something when you drop into a lower dimension, whether you use PCA or just lower the number of neurons in the pre-SVM layer.
Comments.UserDisplayName: Dave
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-09-17T16:26:02.097Z
Comments.Text: This code here, assuming X is your (600, 32786) matrix, reduces your feature space to (600, 250). However, you have to keep in mind that PCA does not remove redundant or less important features so that your are left with the 250 of your most important original features. This is merely a projection of your data onto 250 principal axes. If that's what you wanted to do then yes your code is correct save for the fact that I am not sure why you have performed PCA on `test_images` rather than `X`.
Comments.UserDisplayName: Minuano
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-09-18T01:32:12.427Z
Comments.Text: @Minuano, I was mistaken I should use X instead as you said. Actually I wanted to reduce  the dimension of the feature space while keeping the most relevant features. Is there any solution to do that?
Comments.UserDisplayName: root
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-09-18T08:23:12.673Z
Comments.Text: There are several ways you can go about doing that. Most commonly used methods, as far as I know, involve calculating the correlation of your variables with each other ( a (32786, 32786) correlation matrix) and keeping only the top 250 most uncorrelated features. You can also calculate the correlation between your features and your target variable and try to remove uncorrelated features. For further insight I encourage you to read this [article](https://yanlinc.medium.com/how-to-do-feature-selection-dimension-reduction-883c844aaaf6).
Comments.UserDisplayName: Minuano
Tags: <convolutional-neural-networks><support-vector-machine>
Title: How to use PCA to reduce the dimension of output features of convolutional layers of CNN
UserDisplayName: root
ThreadText: # How to use PCA to reduce the dimension of output features of convolutional layers of CNN
  
  (Asked by: username_0 on 2022-09-14)
  
  I am working on a hybrid CNN-SVM classification using python. I tried to get the final output features of the CNN model(flatten layer) to be fed to the SVM classifier. The output shape obtained from the flatten layer was (600,32786).For computational reasons, I wanted to reduce the shape of the obtained features to (600,250). I was looking for a way to do that without losing relevant features, and I read about the PCA technique.
  I saved the output features in a(.npy) file. I tried the following:
  X= np.load('features.npy')
  pca_250 = PCA(n_components=250)
  pca_250.fit(test_images)
  Y = pca_250.transform(test_images)
  
  Is this correct?
  Regards
  
  **Tags:** convolutional-neural-networks, support-vector-machine
  > **Comments:**
  > **username_1**: You’re basically guaranteed to lose something when you drop into a lower dimension, whether you use PCA or just lower the number of neurons in the pre-SVM layer.  
  > **username_2**: This code here, assuming X is your (600, 32786) matrix, reduces your feature space to (600, 250). However, you have to keep in mind that PCA does not remove redundant or less important features so that your are left with the 250 of your most important original features. This is merely a projection of your data onto 250 principal axes. If that's what you wanted to do then yes your code is correct save for the fact that I am not sure why you have performed PCA on `test_images` rather than `X`.  
  > **username_0**: @Minuano, I was mistaken I should use X instead as you said. Actually I wanted to reduce  the dimension of the feature space while keeping the most relevant features. Is there any solution to do that?  
  > **username_2**: There are several ways you can go about doing that. Most commonly used methods, as far as I know, involve calculating the correlation of your variables with each other ( a (32786, 32786) correlation matrix) and keeping only the top 250 most uncorrelated features. You can also calculate the correlation between your features and your target variable and try to remove uncorrelated features. For further insight I encourage you to read this [article](https://yanlinc.medium.com/how-to-do-feature-selection-dimension-reduction-883c844aaaf6).  
  
  ---
  
  ## 0 answers
  
  