Which exact loss do we minimize in a VAE model? Reading about VAEs here and there, I often get stuck in the confusion about which quantity gets minimized as VAE objective.
After some calculations, here's what we get at:
$\log p_\theta(x) \ge - \mathbb{E}_{q_\phi}[\log \, q_\phi(z|x)] + \mathbb{E}_{q_\phi}[\log \, p_{\theta}(z,x)] $
The component on the right is what is commonly known as ELBO. It can be rewritten as:
$$ELBO(\phi) = \mathbb{E}_{q_\phi}[\log \, p_\theta(x|z)] - \mathbb{E}_{q_\phi} \left[\log \, \frac{q_\phi (z|x)}{p_\theta(z)}\right]$$
This quantity is maximixed (from what I've understood). Does this mean that in the concrete neural network implementations we need to provde $-ELBO(\phi)$ as loss function?
note: for simplicity i use $\theta$ interchangebly above even though they might be different.

Yes, maximizing the ELBO is equivalent to minimizing the negative ELBO. This is a sign convention. You minimize the negative ELBO (also called the variational free energy) in the standard training objective for a variational autoencoder.
