Variational inference : is evidence constant? I'm studying variational inference (in the context of VAEs), and I'm watching this video at this time point. At this point in the video, the goal of approximating the intractable posterior $p_{\theta}(z|x)$ by an approximate posterior $q_{\phi}(z|x)$ has been set, and the ELBO formulation has been presented.

As is shown on the image, what the author of the video says at this point is that, since marginal log-likelihood will not change, maximizing ELBO minimizes the KL divergence between the $q$ and $p$ distributions (which is what we want). To quote more precisely, the author says : "We cannot compute [marginal log-likelihood] but we know that it will not change and so we have found a way around our KL divergence".
This sentence confused me. In the context of variational auto-encoders for generating data, I thought that the goal was specifically to increase the evidence by finding the optimal $\theta$ (and $\phi$) parameters, and so that evidence $p_{\theta}(x)$ will be maximized during the optimisation process (and I guess, $D_{KL}(q_{\phi}||p_{\theta})$ decreases also, but sort of as a by-product ?). So does evidence change or not during the training process ?
Side-question : I think I'm getting confused about some very basic statistical notations as they are sometimes
inconsistent between different materials. Is it true that usually, a notation such as $p(X)$ is a distribution, and $p(x)$ is a number representing the probability $p(X=x)$ ? So evidence is a number, right, not a distribution ?

In the context of VAE, you have two moving parts - $q_\phi(z|x)$ and $p_\theta(x|z)$. You only increase the evidence by optimizing the $\theta$'s (the weights of the decoder) - basically doing maximum likelihood. You do this without any regard to VI, or to $\phi$. You then find the encoder by doing VI, and optimizing the encoder weights.

So, does evidence change or not during the training process ?

It does, but it's done regardless of VI. So, the above video explanation does not relate to it at all.
The evidence is a function that depends on the data. For a given dataset it will be a normalizing constant / number.
