Answers.Body: <p>Just as completing, in general, the autoencoders are an <em>unsupervised learning technique</em> in which we use neural networks for the task of representation learning. Specifically, we'll design a neural network architecture to impose a network bottleneck, forcing a <em>compressed knowledge representation</em> of the original input. In this case, we dont generate any new data and just compress them.
  In the case of VQ-VAE or VAE, we generate new data because the architecture encodes all data in latent space and samples from that (continuous in VAE and discrete in VQ-VAE). Therefore, although the decoder will generate similar outputs (if regulated well), the input data is not just a compressed version.For better understanding, in VAE, the mean and variance of encoded data are used for generation.</p>
  <p>Also, you can read these good references: <a href="https://www.jeremyjordan.me/autoencoders/" rel="nofollow noreferrer">https://www.jeremyjordan.me/autoencoders/</a> , <a href="https://www.v7labs.com/blog/autoencoders-guide" rel="nofollow noreferrer">https://www.v7labs.com/blog/autoencoders-guide</a> ,  <a href="https://www.analyticsvidhya.com/blog/2021/06/complete-guide-on-how-to-use-autoencoders-in-python/" rel="nofollow noreferrer">https://www.analyticsvidhya.com/blog/2021/06/complete-guide-on-how-to-use-autoencoders-in-python/</a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-07T14:55:12.220Z
Answers.Text: As it’s currently written, your answer is unclear. Please [edit] to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers [in the help center](/help/how-to-answer).
Answers.UserDisplayName: Community
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-07T14:32:18.237Z
Answers.LastActivityDate: 2022-07-07T15:29:20.080Z
Answers.LastEditDate: 2022-07-07T15:29:20.080Z
Answers.UserDisplayName: Pouyan
Answers.Body: <p>An autoencoder is not considered a generative model, because it only reconstructs the given input. You could use the decoder <em>like</em> a generative model by putting in different vectors. However, the standard autoencoder mostly learns a sparse latent space. This means that you will have distinct clusters in the latent space (see the left image below). The decoder has never learned to reconstruct vectors in between the clusters, so it will produce very abstract things - mostly garbage.</p>
  <p>Instead a variational autoencoder (VAE) is considered a generative model. It's basically an autoencoder with a modified bottleneck. This VAE learns a dense latent space (see image on the right), this means you can sample any vector from the latent space, pass it to the model and it will give you a nice result with somewhat interpolated object properties from the dataset.</p>
  <p><a href="https://news.sophos.com/en-us/2018/06/15/using-variational-autoencoders-to-learn-variations-in-data/" rel="noreferrer">This article</a> provides a nice overview of the two models.</p>
  <p><a href="https://i.sstatic.net/tIvHU.png" rel="noreferrer"><img src="https://i.sstatic.net/tIvHU.png" alt="latent_spaces" /></a></p>
  <p><em>Figure taken from <a href="https://news.sophos.com/en-us/2018/06/15/using-variational-autoencoders-to-learn-variations-in-data/" rel="noreferrer">here</a></em></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-06-29T12:46:43.713Z
Answers.LastActivityDate: 2022-06-29T12:46:43.713Z
Answers.UserDisplayName: Chillston
Body: <p>I am wondering how a plain auto encoder is a generative model though its version might be but how can a plain auto encoder can be generative. I know that Vaes which is a version of the autoencoder is generative as it generates distribution for latent variables and whole data explicitly. But I am not able to think how an autoencoder generates probability distribution and becomes a generative model.</p>
  <p>Also from this youtube video: <a href="https://www.youtube.com/watch?v=c27SHdQr4lw&amp;t=380s&amp;ab_channel=PaulHand" rel="noreferrer">here</a> It says plain auto encoder is not a generative model. See last line from picture.</p>
  <p><a href="https://i.sstatic.net/28Eq4.png" rel="noreferrer"><img src="https://i.sstatic.net/28Eq4.png" alt="enter image description here" /></a></p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-04-23T15:21:24.950Z
Comments.Text: I think one of the reasons that plain AE is not generative is because it does not learn a joint probability function. For exemple, VAE learns p(x,z), which z is the latent variable. Also, the main goal of VAE is not reduce the reconstruction error, but estimate the intractable posterior distribution of z, p(z|x).
Comments.UserDisplayName: Rafael Toledo
Tags: <deep-learning><autoencoders><generative-model>
Title: Is plain autoencoder a generative model?
UserDisplayName: Nervous Hero
ThreadText: # Is plain autoencoder a generative model?
  
  (Asked by: username_0 on 2022-06-29)
  
  I am wondering how a plain auto encoder is a generative model though its version might be but how can a plain auto encoder can be generative. I know that Vaes which is a version of the autoencoder is generative as it generates distribution for latent variables and whole data explicitly. But I am not able to think how an autoencoder generates probability distribution and becomes a generative model.
  
  Also from this youtube video: [here](https://www.youtube.com/watch?v=c27SHdQr4lw&t=380s&ab_channel=PaulHand) It says plain auto encoder is not a generative model. See last line from picture.
  
  [![enter image description here](https://i.sstatic.net/28Eq4.png)](https://i.sstatic.net/28Eq4.png)
  
  **Tags:** deep-learning, autoencoders, generative-model
  
  > **Comments:**
  > **username_1**: I think one of the reasons that plain AE is not generative is because it does not learn a joint probability function. For exemple, VAE learns p(x,z), which z is the latent variable. Also, the main goal of VAE is not reduce the reconstruction error, but estimate the intractable posterior distribution of z, p(z|x).  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_4 on 2022-06-29. Score: 12 | Accepted answer
  
  An autoencoder is not considered a generative model, because it only reconstructs the given input. You could use the decoder *like* a generative model by putting in different vectors. However, the standard autoencoder mostly learns a sparse latent space. This means that you will have distinct clusters in the latent space (see the left image below). The decoder has never learned to reconstruct vectors in between the clusters, so it will produce very abstract things - mostly garbage.
  
  Instead a variational autoencoder (VAE) is considered a generative model. It's basically an autoencoder with a modified bottleneck. This VAE learns a dense latent space (see image on the right), this means you can sample any vector from the latent space, pass it to the model and it will give you a nice result with somewhat interpolated object properties from the dataset.
  
  [This article](https://news.sophos.com/en-us/2018/06/15/using-variational-autoencoders-to-learn-variations-in-data/) provides a nice overview of the two models.
  
  [![latent_spaces](https://i.sstatic.net/tIvHU.png)](https://i.sstatic.net/tIvHU.png)
  
  *Figure taken from [here](https://news.sophos.com/en-us/2018/06/15/using-variational-autoencoders-to-learn-variations-in-data/)*
  
  
  ---
  
  ### Answer by username_2 on 2022-07-07. Score: 0
  
  Just as completing, in general, the autoencoders are an *unsupervised learning technique* in which we use neural networks for the task of representation learning. Specifically, we'll design a neural network architecture to impose a network bottleneck, forcing a *compressed knowledge representation* of the original input. In this case, we dont generate any new data and just compress them.
  In the case of VQ-VAE or VAE, we generate new data because the architecture encodes all data in latent space and samples from that (continuous in VAE and discrete in VQ-VAE). Therefore, although the decoder will generate similar outputs (if regulated well), the input data is not just a compressed version.For better understanding, in VAE, the mean and variance of encoded data are used for generation.
  
  Also, you can read these good references: <https://www.jeremyjordan.me/autoencoders/> , <https://www.v7labs.com/blog/autoencoders-guide> , <https://www.analyticsvidhya.com/blog/2021/06/complete-guide-on-how-to-use-autoencoders-in-python/>
  
  > **Comments:**
  > **username_3**: As it’s currently written, your answer is unclear. Please [edit] to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers [in the help center](/help/how-to-answer).  
  
  ---
  
  