Answers.Body: <p>It seems that you think that we want to perform classification with VAEs or that images that we pass to the encoder fall into more than one category. The other answer already points out that VAEs are not typically used for classification but for generation tasks, so let me try to answer the main question.</p>
  <p>The <strong>variational auto-encoder (VAE)</strong> and the (deterministic) <strong>auto-encoder</strong> both have an encoder and a decoder and they both convert the inputs to a latent representation, but their inner workings are different: a VAE is a <a href="https://en.wikipedia.org/wiki/Generative_model" rel="nofollow noreferrer">generative</a> <a href="https://ai.stackexchange.com/a/12354/2444">statistical model</a>, while the AE can be viewed just as a data compressor (and decompressor).</p>
  <p>In an <strong>AE</strong>, given an input <span class="math-container">$\mathbf{x}$</span> (e.g. an image), the encoder produces <strong>one</strong> latent vector <span class="math-container">$\mathbf{z_x}$</span>, which can be decoded into <span class="math-container">$\mathbf{\hat{x}}$</span> (another image which should be similar or related to <span class="math-container">$\mathbf{x}$</span>). Compactly, this can be presented as <span class="math-container">$\mathbf{\hat{x}}=f(\mathbf{z_x}=g(\mathbf{x}))$</span>, where <span class="math-container">$g$</span> is the encoder and <span class="math-container">$f$</span> is the decoder. This operation is deterministic: so, given the same <span class="math-container">$\mathbf{x}$</span>, the same <span class="math-container">$\mathbf{z_x}$</span> and <span class="math-container">$\mathbf{\hat{x}}$</span> are produced.</p>
  <p>In a <strong>VAE</strong>, given an input <span class="math-container">$\mathbf{x} \in X$</span> (e.g. an image), more than one latent vector, <span class="math-container">$\mathbf{z_{x}}^i \in Z$</span>, can be produced, because the encoder attempts to learn the probability distribution <span class="math-container">$q_\phi(z \mid x)$</span>, which can be e.g. <span class="math-container">$\mathcal{N}(\mu, \sigma)$</span>, which we can sample from, where <span class="math-container">$\mu, \sigma = g_\theta(\mathbf{x})$</span>. In practice, <span class="math-container">$g_\theta$</span> is a neural network with weights <span class="math-container">$\phi$</span>. We can sample latent vectors <span class="math-container">$\mathbf{z_{x}}^i$</span> from <span class="math-container">$\mathcal{N}(\mu, \sigma)$</span>, which should be &quot;good&quot; representations of a given <span class="math-container">$\mathbf{x}$</span>.</p>
  <p><strong>Why is it useful to learn <span class="math-container">$q_\phi(z \mid x)$</span>?</strong> There are many uses cases. For example, given multiple corrupted/noisy versions of an image, you can reconstruct the original uncorrupted image. However, note that you can use the <strong>AE</strong> also for denoising. <a href="https://www.tensorflow.org/tutorials/generative/autoencoder" rel="nofollow noreferrer">Here</a> you have a TensorFlow example that illustrates this. The difference is is that, again, given the same noisy image, the model will always produce the same reconstructed image. You can also use the VAE for drug design <a href="http://proceedings.mlr.press/v80/jin18a/jin18a.pdf" rel="nofollow noreferrer">[1]</a>. See also <a href="https://ai.stackexchange.com/q/11405/2444">this post</a>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-06-25T00:19:35.480Z
Answers.Text: thanks, i thought those are same. Are they also implemented differently?
Answers.UserDisplayName: user8426627
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-06-29T20:18:24.470Z
Answers.Text: @user8426627 They have a similar implementation because both have an encoder and a decoder, but the encoder, in the case of the VAE, produces a mean and variance, rather than just a latent vector.
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-06-29T21:04:32.493Z
Answers.Text: that sounds like input is interpreted differently (input preparation is not part of nn) but the network itself has same architecture
Answers.UserDisplayName: user8426627
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-06-29T21:08:59.377Z
Answers.Text: @user8426627 It is not the input, it is more the output. The mean and the variance will be the output of some NN. However, both in the case of AE and VAE, at least conceptually, there is an encoder and a decoder. In this way, they are similar.
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-05-24T11:08:47.010Z
Answers.LastActivityDate: 2022-06-04T14:06:46.513Z
Answers.LastEditDate: 2022-06-04T14:06:46.513Z
Answers.UserDisplayName: nbro
Body: <p><strong>What is the advantage of using a VAE over a deterministic auto-encoder?</strong></p>
  <p>For example, assuming we have just 2 labels, a deterministic auto-encoder will always map a given image to the same latent vector. However, one expects that after the training, the 2 classes will form separate clusters in the encoder space.</p>
  <p>In the case of the VAE, an image is mapped to an encoding vector probabilistically. However, one still ends up with 2 separate clusters. Now, if one passes a new image (at the test time), in both cases the network should be able to place that new image in one of the 2 clusters.</p>
  <p>How are these 2 clusters created using the VAE better than the ones from the deterministic case?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-05-24T09:51:59.467Z
Comments.Text: "However, one expects that after the training, the 2 classes will form separate clusters in the encoder space.", what do you mean by "the 2 classes"?
Comments.UserDisplayName: nbro
Tags: <neural-networks><applications><autoencoders><variational-autoencoder>
Title: What is the advantage of using a VAE over a deterministic auto-encoder?
UserDisplayName: Alex Marshall
ThreadText: # What is the advantage of using a VAE over a deterministic auto-encoder?
  
  (Asked by: username_0 on 2019-05-24)
  
  **What is the advantage of using a VAE over a deterministic auto-encoder?**
  
  For example, assuming we have just 2 labels, a deterministic auto-encoder will always map a given image to the same latent vector. However, one expects that after the training, the 2 classes will form separate clusters in the encoder space.
  
  In the case of the VAE, an image is mapped to an encoding vector probabilistically. However, one still ends up with 2 separate clusters. Now, if one passes a new image (at the test time), in both cases the network should be able to place that new image in one of the 2 clusters.
  
  How are these 2 clusters created using the VAE better than the ones from the deterministic case?
  
  **Tags:** neural-networks, applications, autoencoders, variational-autoencoder
  > **Comments:**
  > **username_1**: "However, one expects that after the training, the 2 classes will form separate clusters in the encoder space.", what do you mean by "the 2 classes"?  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-05-24. Score: 1
  
  It seems that you think that we want to perform classification with VAEs or that images that we pass to the encoder fall into more than one category. The other answer already points out that VAEs are not typically used for classification but for generation tasks, so let me try to answer the main question.
  
  The **variational auto-encoder (VAE)** and the (deterministic) **auto-encoder** both have an encoder and a decoder and they both convert the inputs to a latent representation, but their inner workings are different: a VAE is a [generative](https://en.wikipedia.org/wiki/Generative_model) [statistical model](https://ai.stackexchange.com/a/12354/2444), while the AE can be viewed just as a data compressor (and decompressor).
  
  In an **AE**, given an input $\mathbf{x}$ (e.g. an image), the encoder produces **one** latent vector $\mathbf{z\_x}$, which can be decoded into $\mathbf{\hat{x}}$ (another image which should be similar or related to $\mathbf{x}$). Compactly, this can be presented as $\mathbf{\hat{x}}=f(\mathbf{z\_x}=g(\mathbf{x}))$, where $g$ is the encoder and $f$ is the decoder. This operation is deterministic: so, given the same $\mathbf{x}$, the same $\mathbf{z\_x}$ and $\mathbf{\hat{x}}$ are produced.
  
  In a **VAE**, given an input $\mathbf{x} \in X$ (e.g. an image), more than one latent vector, $\mathbf{z\_{x}}^i \in Z$, can be produced, because the encoder attempts to learn the probability distribution $q\_\phi(z \mid x)$, which can be e.g. $\mathcal{N}(\mu, \sigma)$, which we can sample from, where $\mu, \sigma = g\_\theta(\mathbf{x})$. In practice, $g\_\theta$ is a neural network with weights $\phi$. We can sample latent vectors $\mathbf{z\_{x}}^i$ from $\mathcal{N}(\mu, \sigma)$, which should be "good" representations of a given $\mathbf{x}$.
  
  **Why is it useful to learn $q\_\phi(z \mid x)$?** There are many uses cases. For example, given multiple corrupted/noisy versions of an image, you can reconstruct the original uncorrupted image. However, note that you can use the **AE** also for denoising. [Here](https://www.tensorflow.org/tutorials/generative/autoencoder) you have a TensorFlow example that illustrates this. The difference is is that, again, given the same noisy image, the model will always produce the same reconstructed image. You can also use the VAE for drug design [[1]](http://proceedings.mlr.press/v80/jin18a/jin18a.pdf). See also [this post](https://ai.stackexchange.com/q/11405/2444).
  
  > **Comments:**
  > **username_2**: thanks, i thought those are same. Are they also implemented differently?  
  > **username_1**: @user8426627 They have a similar implementation because both have an encoder and a decoder, but the encoder, in the case of the VAE, produces a mean and variance, rather than just a latent vector.  
  > **username_2**: that sounds like input is interpreted differently (input preparation is not part of nn) but the network itself has same architecture  
  > **username_1**: @user8426627 It is not the input, it is more the output. The mean and the variance will be the output of some NN. However, both in the case of AE and VAE, at least conceptually, there is an encoder and a decoder. In this way, they are similar.  
  
  ---
  
  