<p>Some examples of dimensionality reduction techniques:</p>
<div class="s-table-container">
<table class="s-table">
<thead>
<tr>
<th>Linear methods</th>
<th>Non-linear methods</th>
<th>Graph-based methods<br>(&quot;Network embedding&quot;)</th>
</tr>
</thead>
<tbody>
<tr>
<td>PCA<br> CCA<br> ICA<br> SVD<br> LDA<br> NMF</td>
<td>Kernel PCA<br> GDA<br> Autoencoders<br> t-SNE<br> UMAP<br> MVU</td>
<td>Diffusion maps<br> Graph Autoencoders<br><br> Graph-based kernel PCA<br> (Isomap, LLE, Hessian LLE, Laplacian Eigenmaps)</td>
</tr>
</tbody>
</table>
</div>
<p>Though there are <a href="https://en.wikipedia.org/wiki/Nonlinear_dimensionality_reduction" rel="nofollow noreferrer">many more</a>.</p>

CC BY-SA 4.0
2020-04-27T21:43:06.830Z
2021-02-16T13:54:04.610Z
2021-02-16T13:54:04.610Z
brazofuerte
<p>Dimensionality reduction could be achieved by using an <strong>Autoencoder</strong> Network, which learns a representation (or Encoding) for the input data. While training, the reduction side (Encoder) reduces the data to a lower-dimension and a reconstructing side (Decoder) tries to reconstruct the original input from the intermediate reduced encoding. </p>

<p>You could assign the encoder layer output (<span class="math-container">$L_i$</span>) to a desired dimension (lower than that of the input). Once trained, <span class="math-container">$L_i$</span> could be used as a alternative representation of your input data in a lower feature-space, and can be used for further computations.</p>

<p><a href="https://i.sstatic.net/kLtGc.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/kLtGc.png" alt="Autoencoder Architecture"></a></p>

CC BY-SA 4.0
2020-01-26T23:28:10.927Z
Maybe this answer could be further improved if you link to a paper or implementation that shows the application of AE to reduce the dimensionality of feature vectors. If you consider images feature vectors, then, in a way, AE are commonly applied to reduce the dimensionality of images (or feature vectors), but what if the inputs are not images?
nbro
CC BY-SA 4.0
2020-02-03T06:44:42.500Z
Is there any python library perform autoencoder efficiently?
doplano
CC BY-SA 4.0
2020-02-03T21:18:47.780Z
@FäridAlijani Not that I know of. However, designing one in Keras wouldn't be much of a task. The following Keras blog might help : 
https://blog.keras.io/building-autoencoders-in-keras.html
s_bh
CC BY-SA 4.0
2020-01-26T23:01:58.050Z
2020-01-26T23:09:52.010Z
2020-01-26T23:09:52.010Z
s_bh
<p>Given a pre-trained CNN model, I extract feature vector of images in reference and query dataset with several thousands of elements.</p>

<p>I would like to apply some augmentation techniques to reduce the feature vector dimension to speed up cosine similarity/euclidean distance matrix calculation.</p>

<p>I have already come up with the following two methods in my literature review:</p>

<ol>
<li>Principal Component Analysis (PCA) + Whitening</li>
<li>Locality Search Hashing (LSH)</li>
</ol>

<p>Are there more approaches to perform dimensionality reduction of feature vectors? If so, what are the pros/cons of each perhaps?</p>

<convolutional-neural-networks><representation-learning><principal-component-analysis><dimensionality-reduction>
What are examples of approaches to dimensionality reduction of feature vectors?
doplano
# What are examples of approaches to dimensionality reduction of feature vectors?

(Asked by: username_0 on 2020-01-23)

Given a pre-trained CNN model, I extract feature vector of images in reference and query dataset with several thousands of elements.

I would like to apply some augmentation techniques to reduce the feature vector dimension to speed up cosine similarity/euclidean distance matrix calculation.

I have already come up with the following two methods in my literature review:

1. Principal Component Analysis (PCA) + Whitening
2. Locality Search Hashing (LSH)

Are there more approaches to perform dimensionality reduction of feature vectors? If so, what are the pros/cons of each perhaps?

**Tags:** convolutional-neural-networks, representation-learning, principal-component-analysis, dimensionality-reduction


---

## 2 answers

### Answer by username_2 on 2020-01-26. Score: 4 | Accepted answer

Dimensionality reduction could be achieved by using an **Autoencoder** Network, which learns a representation (or Encoding) for the input data. While training, the reduction side (Encoder) reduces the data to a lower-dimension and a reconstructing side (Decoder) tries to reconstruct the original input from the intermediate reduced encoding.

You could assign the encoder layer output ($L\_i$) to a desired dimension (lower than that of the input). Once trained, $L\_i$ could be used as a alternative representation of your input data in a lower feature-space, and can be used for further computations.

[![Autoencoder Architecture](https://i.sstatic.net/kLtGc.png)](https://i.sstatic.net/kLtGc.png)

> **Comments:**
> **username_3**: Maybe this answer could be further improved if you link to a paper or implementation that shows the application of AE to reduce the dimensionality of feature vectors. If you consider images feature vectors, then, in a way, AE are commonly applied to reduce the dimensionality of images (or feature vectors), but what if the inputs are not images?  
> **username_0**: Is there any python library perform autoencoder efficiently?  
> **username_2**: @FäridAlijani Not that I know of. However, designing one in Keras wouldn't be much of a task. The following Keras blog might help :  https://blog.keras.io/building-autoencoders-in-keras.html  

---

### Answer by username_1 on 2020-04-27. Score: 2

Some examples of dimensionality reduction techniques:

| Linear methods | Non-linear methods | Graph-based methods ("Network embedding") |
| --- | --- | --- |
| PCA  CCA  ICA  SVD  LDA  NMF | Kernel PCA  GDA  Autoencoders  t-SNE  UMAP  MVU | Diffusion maps  Graph Autoencoders   Graph-based kernel PCA  (Isomap, LLE, Hessian LLE, Laplacian Eigenmaps) |

Though there are [many more](https://en.wikipedia.org/wiki/Nonlinear_dimensionality_reduction).


---

