Answers.Body: <p>According to various experimentation on autoencoders, it is very possible to have latent vector of size 1. Various layers can help the downsizing of the original input to a very small size of 1. But an issue may arise during decoding. If you're expecting that through one or two or maybe five layers in decoder you can achieve an accurate reconstruction, it is highly unlikely abd the result will turn out to be blurry. Maybe a great network with various parameters may help the reconstruction without considering generalization as asked by you.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-05-19T11:31:50.793Z
Answers.Text: Hi and welcome to AI SE! Thanks for trying to contribute. You say "According to various experimentation on autoencoders". Can you cite some work that goes into this direction?
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-05-19T12:27:35.803Z
Answers.Text: The answer is completely in competence with my own experimentation. It is not published anywhere I guess. But, I may post a link: https://towardsdatascience.com/how-to-make-an-autoencoder-2f2d99cd5103 and here, if you tweak the network and latent size, you can have a look into the changes.
Answers.UserDisplayName: Anubhav Sachan
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-05-19T11:01:14.230Z
Answers.LastActivityDate: 2020-05-19T12:53:44.570Z
Answers.LastEditDate: 2020-05-19T12:53:44.570Z
Answers.UserDisplayName: Anubhav Sachan
Body: <p>Given e.g. 1M vectors of <span class="math-container">$1000$</span> floating points each, where every point in vectors is sampled from a uniform distribution between <span class="math-container">$-1$</span> to <span class="math-container">$1$</span>:</p>
  
  <p>Is it possible to have the bottleneck of the AE network with size 1? In other words, without caring about generalization, is it possible to train a network, where, given only 1 encoded value, it can recreate any of the 1M examples?</p>
  
Tags: <neural-networks><machine-learning><autoencoders><information-theory>
Title: Is it possible to have the latent vector of an auto-encoder with size 1?
UserDisplayName: ENECO
ThreadText: # Is it possible to have the latent vector of an auto-encoder with size 1?
  
  (Asked by: username_0 on 2020-05-19)
  
  Given e.g. 1M vectors of $1000$ floating points each, where every point in vectors is sampled from a uniform distribution between $-1$ to $1$:
  
  Is it possible to have the bottleneck of the AE network with size 1? In other words, without caring about generalization, is it possible to train a network, where, given only 1 encoded value, it can recreate any of the 1M examples?
  
  **Tags:** neural-networks, machine-learning, autoencoders, information-theory
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-05-19. Score: 1
  
  According to various experimentation on autoencoders, it is very possible to have latent vector of size 1. Various layers can help the downsizing of the original input to a very small size of 1. But an issue may arise during decoding. If you're expecting that through one or two or maybe five layers in decoder you can achieve an accurate reconstruction, it is highly unlikely abd the result will turn out to be blurry. Maybe a great network with various parameters may help the reconstruction without considering generalization as asked by you.
  
  > **Comments:**
  > **username_2**: Hi and welcome to AI SE! Thanks for trying to contribute. You say "According to various experimentation on autoencoders". Can you cite some work that goes into this direction?  
  > **username_1**: The answer is completely in competence with my own experimentation. It is not published anywhere I guess. But, I may post a link: https://towardsdatascience.com/how-to-make-an-autoencoder-2f2d99cd5103 and here, if you tweak the network and latent size, you can have a look into the changes.  
  
  ---
  
  