<p>I don't quite understand why, in <em>Conditional Variational Autoencoder (CVAE)</em>, we concatenate a conditioning vector two times, at encoder and decoder respectively.
After we concatenate it once at the beginning, isn't the latent distribution going to already incorporate knowledge about the label?</p>
<p>I know that, from a practical perspective we need to concatenate it at the decoder as well, since we want to be able then to generate new instances associating a specific label, but I'm missing more the mathematical motivations for which we concatenate it two times.</p>
<p>To be more precise, let's consider how the  objective function of <em>CVAE</em> is defined:</p>
<p><span class="math-container">$$\mathcal{L}_{CVAE} = \mathbb{E}_\textbf{z}[log \, p_\theta(\textbf{x}|\textbf{z},\textbf{c})] - D_{KL}[q_\phi(\textbf{z}|\textbf{x},\textbf{c})||p(\textbf{z}|\textbf{c})] $$</span></p>
<p><strong>My question:</strong> <em>Why do we condition on <span class="math-container">$\textbf{c}$</span> in both terms?</em></p>

CC BY-SA 4.0
2022-03-06T00:04:35.010Z
To make sure I understand your question, in the objective function, $c$ is what you call _label_? [This paper](https://papers.nips.cc/paper/2015/file/8d55a249e6baa5c06772297520da2051-Paper.pdf) seems to use a different notation for the objective function. It might be a good idea to provide the reference you're using.
nbro
CC BY-SA 4.0
2022-03-06T00:16:56.297Z
Exactly, $\textbf{c}$ is the label in the objective function I have specified
James Arten
CC BY-SA 4.0
2022-03-07T08:59:27.303Z
So, your question is: "In CVAE, why are both the encoder and decoder defined as a function or probability distribution that is conditioned on the label"?
nbro
CC BY-SA 4.0
2022-03-07T09:52:18.560Z
Yes that's pretty much what I wanted to ask
James Arten
CC BY-SA 4.0
2022-03-17T10:04:32.373Z
Ok, can you please provide the source of this equation? Because I'm looking at the original paper, as I already said, and it uses a different notation, and it's harder for me to associate the symbols there with yours. It seems that you're using $c$ to denote what the authors of the paper denote by $x$, and you use $x$ to denote what they denote by $y$, the label or output variable. $z$ is the latent vector both in your case and in the original paper. $c$ seems to be the input to the auto-encoder, not a label.
nbro
CC BY-SA 4.0
2022-03-17T10:05:16.850Z
So, they are not conditioning on a "label" (which would be $y$ in the paper and $x$ in your notation), but conditioning on the inputs. So, please, edit your post to reflect this comment and to either make your notation equal to the original paper's one or to provide a reference that uses this notation. Moreover, your question "Why do we condition both on encoder and decoder?" is not specific and clear enough. Because, even in the original ELBO, you condition on something. You want to ask why we're conditioning on $c$ in both terms.
nbro
CC BY-SA 4.0
2022-03-17T14:14:50.773Z
That's exactly what I'm trying to better frame. Why do we condition on $\textbf{c}$ in both terms of the loss? I've modified the question clearly expressing this doubt.
James Arten
CC BY-SA 4.0
2022-03-17T14:15:59.733Z
I had understood that this was your question, but I recommend that you follow my other suggestions in the comments above.
nbro
CC BY-SA 4.0
2024-04-16T14:39:51.070Z
I suspect that z needs to be conditioned on c to get the KL maffs to work out. Crudely it also makes sense that the encoder can do a better job with more information, in which case the decoder can benefit. But I also expect it sometimes won't matter in practice. Not been able to find work on this specifically. So am going to test.
Fred Shone
<objective-functions><variational-autoencoder><conditional-vae>
In CVAE's objective function, why do both terms condition on $\textbf{c}$?
James Arten
# In CVAE's objective function, why do both terms condition on $\textbf{c}$?

(Asked by: username_0 on 2022-03-05)

I don't quite understand why, in *Conditional Variational Autoencoder (CVAE)*, we concatenate a conditioning vector two times, at encoder and decoder respectively.
After we concatenate it once at the beginning, isn't the latent distribution going to already incorporate knowledge about the label?

I know that, from a practical perspective we need to concatenate it at the decoder as well, since we want to be able then to generate new instances associating a specific label, but I'm missing more the mathematical motivations for which we concatenate it two times.

To be more precise, let's consider how the objective function of *CVAE* is defined:

$$\mathcal{L}\_{CVAE} = \mathbb{E}\_\textbf{z}[log \, p\_\theta(\textbf{x}|\textbf{z},\textbf{c})] - D\_{KL}[q\_\phi(\textbf{z}|\textbf{x},\textbf{c})||p(\textbf{z}|\textbf{c})] $$

**My question:** *Why do we condition on $\textbf{c}$ in both terms?*

**Tags:** objective-functions, variational-autoencoder, conditional-vae

> **Comments:**
> **username_1**: To make sure I understand your question, in the objective function, $c$ is what you call _label_? [This paper](https://papers.nips.cc/paper/2015/file/8d55a249e6baa5c06772297520da2051-Paper.pdf) seems to use a different notation for the objective function. It might be a good idea to provide the reference you're using.  
> **username_0**: Exactly, $\textbf{c}$ is the label in the objective function I have specified  
> **username_1**: So, your question is: "In CVAE, why are both the encoder and decoder defined as a function or probability distribution that is conditioned on the label"?  
> **username_0**: Yes that's pretty much what I wanted to ask  
> **username_1**: Ok, can you please provide the source of this equation? Because I'm looking at the original paper, as I already said, and it uses a different notation, and it's harder for me to associate the symbols there with yours. It seems that you're using $c$ to denote what the authors of the paper denote by $x$, and you use $x$ to denote what they denote by $y$, the label or output variable. $z$ is the latent vector both in your case and in the original paper. $c$ seems to be the input to the auto-encoder, not a label.  
> **username_1**: So, they are not conditioning on a "label" (which would be $y$ in the paper and $x$ in your notation), but conditioning on the inputs. So, please, edit your post to reflect this comment and to either make your notation equal to the original paper's one or to provide a reference that uses this notation. Moreover, your question "Why do we condition both on encoder and decoder?" is not specific and clear enough. Because, even in the original ELBO, you condition on something. You want to ask why we're conditioning on $c$ in both terms.  
> **username_0**: That's exactly what I'm trying to better frame. Why do we condition on $\textbf{c}$ in both terms of the loss? I've modified the question clearly expressing this doubt.  
> **username_1**: I had understood that this was your question, but I recommend that you follow my other suggestions in the comments above.  
> **username_2**: I suspect that z needs to be conditioned on c to get the KL maffs to work out. Crudely it also makes sense that the encoder can do a better job with more information, in which case the decoder can benefit. But I also expect it sometimes won't matter in practice. Not been able to find work on this specifically. So am going to test.  

---

## 0 answers

