Answers.Body: <p>Multivariate Gaussian distribution is usually employed to decode real-valued data and Bernoulli distribution is usually employed to decode binary data in <a href="https://en.wikipedia.org/wiki/Variational_autoencoder#" rel="nofollow noreferrer">VAEs</a> as claimed in your own reference.</p>
  <blockquote>
  <p><span class="math-container">${\displaystyle \ln p_{\theta }(x|z)}$</span> is implemented as <span class="math-container">${\displaystyle -{\frac {1}{2}}\|x-D_{\theta }(z)\|_{2}^{2}}$</span>, since that is, up to an additive constant, what <span class="math-container">${\displaystyle x\sim {\mathcal {N}}(D_{\theta }(z),I)}$</span> yields. That is, we model the distribution of <span class="math-container">${\displaystyle x}$</span> conditional on <span class="math-container">${\displaystyle z}$</span> to be a Gaussian distribution centered on <span class="math-container">${\displaystyle D_{\theta }(z)}$</span>. The distribution of <span class="math-container">${\displaystyle q_{\phi }(z|x)}$</span> and <span class="math-container">${\displaystyle p_{\theta }(z)}$</span> are often also chosen to be Gaussians</p>
  </blockquote>
  <p>In summary Gaussian, though may be limiting in modeling complex data such as GMMs, is often used to derive closed-form loss functions or solutions for real-valued data cases.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-09-02T04:59:31.050Z
Answers.LastActivityDate: 2024-09-02T04:59:31.050Z
Answers.UserDisplayName: cinch
Body: <p>My question concerns the paper arxiv.org/pdf/1312.6114. I want to know why they proposed to use MLP Gaussian decoder with parameters given by the MLP transformation of the z variable as the likelihood model <span class="math-container">$p_{\theta}(x|z)$</span>. If they are using NNs to perform inference on the best set of parameters, why using a Gaussian if it is very limiting in modeling complex data?</p>
  
Tags: <variational-autoencoder><multilayer-perceptrons><normal-distribution>
Title: MLP Gaussian Decoder in VAE
UserDisplayName: piero
ThreadText: # MLP Gaussian Decoder in VAE
  
  (Asked by: username_0 on 2024-08-23)
  
  My question concerns the paper arxiv.org/pdf/1312.6114. I want to know why they proposed to use MLP Gaussian decoder with parameters given by the MLP transformation of the z variable as the likelihood model $p\_{\theta}(x|z)$. If they are using NNs to perform inference on the best set of parameters, why using a Gaussian if it is very limiting in modeling complex data?
  
  **Tags:** variational-autoencoder, multilayer-perceptrons, normal-distribution
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2024-09-02. Score: 0
  
  Multivariate Gaussian distribution is usually employed to decode real-valued data and Bernoulli distribution is usually employed to decode binary data in [VAEs](https://en.wikipedia.org/wiki/Variational_autoencoder#) as claimed in your own reference.
  
  > ${\displaystyle \ln p\_{\theta }(x|z)}$ is implemented as ${\displaystyle -{\frac {1}{2}}\|x-D\_{\theta }(z)\|\_{2}^{2}}$, since that is, up to an additive constant, what ${\displaystyle x\sim {\mathcal {N}}(D\_{\theta }(z),I)}$ yields. That is, we model the distribution of ${\displaystyle x}$ conditional on ${\displaystyle z}$ to be a Gaussian distribution centered on ${\displaystyle D\_{\theta }(z)}$. The distribution of ${\displaystyle q\_{\phi }(z|x)}$ and ${\displaystyle p\_{\theta }(z)}$ are often also chosen to be Gaussians
  
  In summary Gaussian, though may be limiting in modeling complex data such as GMMs, is often used to derive closed-form loss functions or solutions for real-valued data cases.
  
  
  ---
  
  