Answers.Body: <p>Some examples of dimensionality reduction techniques:</p>
  <div class="s-table-container">
  <table class="s-table">
  <thead>
  <tr>
  <th>Linear methods</th>
  <th>Non-linear methods</th>
  <th>Graph-based methods<br>(&quot;Network embedding&quot;)</th>
  </tr>
  </thead>
  <tbody>
  <tr>
  <td>PCA<br> CCA<br> ICA<br> SVD<br> LDA<br> NMF</td>
  <td>Kernel PCA<br> GDA<br> Autoencoders<br> t-SNE<br> UMAP<br> MVU</td>
  <td>Diffusion maps<br> Graph Autoencoders<br><br> Graph-based kernel PCA<br> (Isomap, LLE, Hessian LLE, Laplacian Eigenmaps)</td>
  </tr>
  </tbody>
  </table>
  </div>
  <p>Though there are <a href="https://en.wikipedia.org/wiki/Nonlinear_dimensionality_reduction" rel="nofollow noreferrer">many more</a>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-27T21:43:06.830Z
Answers.LastActivityDate: 2021-02-16T13:54:04.610Z
Answers.LastEditDate: 2021-02-16T13:54:04.610Z
Answers.UserDisplayName: brazofuerte
Answers.Body: <p>Dimensionality reduction could be achieved by using an <strong>Autoencoder</strong> Network, which learns a representation (or Encoding) for the input data. While training, the reduction side (Encoder) reduces the data to a lower-dimension and a reconstructing side (Decoder) tries to reconstruct the original input from the intermediate reduced encoding. </p>
  
  <p>You could assign the encoder layer output (<span class="math-container">$L_i$</span>) to a desired dimension (lower than that of the input). Once trained, <span class="math-container">$L_i$</span> could be used as a alternative representation of your input data in a lower feature-space, and can be used for further computations.</p>
  
  <p><a href="https://i.sstatic.net/kLtGc.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/kLtGc.png" alt="Autoencoder Architecture"></a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-26T23:28:10.927Z
Answers.Text: Maybe this answer could be further improved if you link to a paper or implementation that shows the application of AE to reduce the dimensionality of feature vectors. If you consider images feature vectors, then, in a way, AE are commonly applied to reduce the dimensionality of images (or feature vectors), but what if the inputs are not images?
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-02-03T06:44:42.500Z
Answers.Text: Is there any python library perform autoencoder efficiently?
Answers.UserDisplayName: doplano
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-02-03T21:18:47.780Z
Answers.Text: @FäridAlijani Not that I know of. However, designing one in Keras wouldn't be much of a task. The following Keras blog might help : 
  https://blog.keras.io/building-autoencoders-in-keras.html
Answers.UserDisplayName: s_bh
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-26T23:01:58.050Z
Answers.LastActivityDate: 2020-01-26T23:09:52.010Z
Answers.LastEditDate: 2020-01-26T23:09:52.010Z
Answers.UserDisplayName: s_bh
Body: <p>Given a pre-trained CNN model, I extract feature vector of images in reference and query dataset with several thousands of elements.</p>
  
  <p>I would like to apply some augmentation techniques to reduce the feature vector dimension to speed up cosine similarity/euclidean distance matrix calculation.</p>
  
  <p>I have already come up with the following two methods in my literature review:</p>
  
  <ol>
  <li>Principal Component Analysis (PCA) + Whitening</li>
  <li>Locality Search Hashing (LSH)</li>
  </ol>
  
  <p>Are there more approaches to perform dimensionality reduction of feature vectors? If so, what are the pros/cons of each perhaps?</p>
  
Tags: <convolutional-neural-networks><representation-learning><principal-component-analysis><dimensionality-reduction>
Title: What are examples of approaches to dimensionality reduction of feature vectors?
UserDisplayName: doplano
ThreadText: # What are examples of approaches to dimensionality reduction of feature vectors?
  
  (Asked by: username_0 on 2020-01-23)
  
  Given a pre-trained CNN model, I extract feature vector of images in reference and query dataset with several thousands of elements.
  
  I would like to apply some augmentation techniques to reduce the feature vector dimension to speed up cosine similarity/euclidean distance matrix calculation.
  
  I have already come up with the following two methods in my literature review:
  
  1. Principal Component Analysis (PCA) + Whitening
  2. Locality Search Hashing (LSH)
  
  Are there more approaches to perform dimensionality reduction of feature vectors? If so, what are the pros/cons of each perhaps?
  
  **Tags:** convolutional-neural-networks, representation-learning, principal-component-analysis, dimensionality-reduction
  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2020-01-26. Score: 4 | Accepted answer
  
  Dimensionality reduction could be achieved by using an **Autoencoder** Network, which learns a representation (or Encoding) for the input data. While training, the reduction side (Encoder) reduces the data to a lower-dimension and a reconstructing side (Decoder) tries to reconstruct the original input from the intermediate reduced encoding.
  
  You could assign the encoder layer output ($L\_i$) to a desired dimension (lower than that of the input). Once trained, $L\_i$ could be used as a alternative representation of your input data in a lower feature-space, and can be used for further computations.
  
  [![Autoencoder Architecture](https://i.sstatic.net/kLtGc.png)](https://i.sstatic.net/kLtGc.png)
  
  > **Comments:**
  > **username_3**: Maybe this answer could be further improved if you link to a paper or implementation that shows the application of AE to reduce the dimensionality of feature vectors. If you consider images feature vectors, then, in a way, AE are commonly applied to reduce the dimensionality of images (or feature vectors), but what if the inputs are not images?  
  > **username_0**: Is there any python library perform autoencoder efficiently?  
  > **username_2**: @FäridAlijani Not that I know of. However, designing one in Keras wouldn't be much of a task. The following Keras blog might help :  https://blog.keras.io/building-autoencoders-in-keras.html  
  
  ---
  
  ### Answer by username_1 on 2020-04-27. Score: 2
  
  Some examples of dimensionality reduction techniques:
  
  | Linear methods | Non-linear methods | Graph-based methods ("Network embedding") |
  | --- | --- | --- |
  | PCA  CCA  ICA  SVD  LDA  NMF | Kernel PCA  GDA  Autoencoders  t-SNE  UMAP  MVU | Diffusion maps  Graph Autoencoders   Graph-based kernel PCA  (Isomap, LLE, Hessian LLE, Laplacian Eigenmaps) |
  
  Though there are [many more](https://en.wikipedia.org/wiki/Nonlinear_dimensionality_reduction).
  
  
  ---
  
  