Answers.Body: <p>Whether a discrete or continuous class, you can model it the same.</p>
  <p>Denote the encoder <span class="math-container">$q$</span> and the decoder <span class="math-container">$p$</span>. Recall the variational autoencoder's goal is to minimize the <span class="math-container">$KL$</span> divergence between <span class="math-container">$q$</span> and <span class="math-container">$p$</span>'s posterior. i.e. <span class="math-container">$\min_{\theta, \phi} \ KL(q(z|x;\theta) || p(z|x; \phi))$</span> where <span class="math-container">$\theta$</span> and <span class="math-container">$\phi$</span> parameterize the encoder and decoder respectively. To make this tractable this is generally done by using the Evidence Lower Bound (because it has the same minimum) and parametrizing <span class="math-container">$q$</span> with some form of reparametrization trick to make sampling differentiable.</p>
  <p>Now your goal is to condition the sampling. In other words you are looking for modeling <span class="math-container">$p(x|z, c;\phi)$</span> and in turn will once again require <span class="math-container">$q(z|x, c; \theta)$</span>. Your goal will now intuitively become once again <span class="math-container">$\min_{\theta, \phi} \ KL(q(z|x, c;\theta) || p(z|x, c; \phi))$</span>. This is still simply transformed into the ELBO for tractability purposes. In other words your loss becomes <span class="math-container">$E_q[log \ p(x|z,c)] - KL(q(z|x,c)||p(z|c)$</span>.</p>
  <p><strong>Takeaway:</strong> Conditioning doesn't change much, just embed your context and inject it both into the encoder and decoder, the fact that its continuous doesn't change anything. For implementation details, normally people just project/normalize and concatenate it somehow to some representation of <span class="math-container">$x$</span> in both the decoder/encoder.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-07-31T12:49:24.410Z
Answers.LastActivityDate: 2020-11-22T17:51:35.043Z
Answers.LastEditDate: 2020-11-22T17:51:35.043Z
Answers.UserDisplayName: mshlis
Body: <p>The <a href="https://wiseodd.github.io/techblog/2016/12/17/conditional-vae/" rel="nofollow noreferrer">Conditional Variational Autoencoder (CVAE)</a>, introduced in the paper <a href="https://papers.nips.cc/paper/5775-learning-structured-output-representation-using-deep-conditional-generative-models.pdf" rel="nofollow noreferrer">Learning Structured Output Representation using Deep Conditional Generative Models</a> (2015), is an extension of <a href="https://arxiv.org/abs/1312.6114" rel="nofollow noreferrer">Variational Autoencoder (VAE)</a> (2013). In VAEs, we have no control over the data generation process, something problematic if we want to generate some specific data. Say, in MNIST, generate instances of 6.</p>
  <p>So far, I have only been able to find CVAEs that can condition to discrete features (classes). Is there a CVAE that allows us to condition to continuous variables, kind of a stochastic predictive model?</p>
  
Tags: <neural-networks><reference-request><autoencoders><variational-autoencoder><conditional-vae>
Title: Is there a continuous conditional variational auto-encoder?
UserDisplayName: D1X
ThreadText: # Is there a continuous conditional variational auto-encoder?
  
  (Asked by: username_0 on 2019-07-31)
  
  The [Conditional Variational Autoencoder (CVAE)](https://wiseodd.github.io/techblog/2016/12/17/conditional-vae/), introduced in the paper [Learning Structured Output Representation using Deep Conditional Generative Models](https://papers.nips.cc/paper/5775-learning-structured-output-representation-using-deep-conditional-generative-models.pdf) (2015), is an extension of [Variational Autoencoder (VAE)](https://arxiv.org/abs/1312.6114) (2013). In VAEs, we have no control over the data generation process, something problematic if we want to generate some specific data. Say, in MNIST, generate instances of 6.
  
  So far, I have only been able to find CVAEs that can condition to discrete features (classes). Is there a CVAE that allows us to condition to continuous variables, kind of a stochastic predictive model?
  
  **Tags:** neural-networks, reference-request, autoencoders, variational-autoencoder, conditional-vae
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-07-31. Score: 1
  
  Whether a discrete or continuous class, you can model it the same.
  
  Denote the encoder $q$ and the decoder $p$. Recall the variational autoencoder's goal is to minimize the $KL$ divergence between $q$ and $p$'s posterior. i.e. $\min\_{\theta, \phi} \ KL(q(z|x;\theta) || p(z|x; \phi))$ where $\theta$ and $\phi$ parameterize the encoder and decoder respectively. To make this tractable this is generally done by using the Evidence Lower Bound (because it has the same minimum) and parametrizing $q$ with some form of reparametrization trick to make sampling differentiable.
  
  Now your goal is to condition the sampling. In other words you are looking for modeling $p(x|z, c;\phi)$ and in turn will once again require $q(z|x, c; \theta)$. Your goal will now intuitively become once again $\min\_{\theta, \phi} \ KL(q(z|x, c;\theta) || p(z|x, c; \phi))$. This is still simply transformed into the ELBO for tractability purposes. In other words your loss becomes $E\_q[log \ p(x|z,c)] - KL(q(z|x,c)||p(z|c)$.
  
  **Takeaway:** Conditioning doesn't change much, just embed your context and inject it both into the encoder and decoder, the fact that its continuous doesn't change anything. For implementation details, normally people just project/normalize and concatenate it somehow to some representation of $x$ in both the decoder/encoder.
  
  
  ---
  
  