Understanding the Evidence Lower Bound (ELBO) I am reading this tutorial about Variational Inference, which includes the following depiction of ELBO as the lower bound on log-likelihood on the third page. In the tutorial, $x_i$ is the observed data and $z_i$ is the latent variable.

Let me first ask a quick clarification question. ELBO is defined in equation (4) as $\mathscr{L}(q, \theta) := -\operatorname*{KL}(q(z_i)||\mathbb{P}(x_i, z_i|\theta)).$ Since the KL divergence is non-negative, the ELBO must be non-positive. Am I right? The illustration seems to indicate that ELBO can be positive, so I am a little confused.
Now the main question: I understand ELBO is a lower bound of the log-likelihood so we want to make it as large as possible. However, why can we ignore the other term, $\operatorname*{KL}(q(z_i)||\mathbb{P}(z_i | x_i, \theta))$? In other words, how do we ensure increasing ELBO doesn't accidentally decrease $\operatorname*{KL}(q(z_i)||\mathbb{P}(z_i | x_i, \theta))$, given that they share the same $\theta$?

Don’t forget that log probabilities are non-positive, too. They’re the logs of values that cannot exceed 1.
With that in mind, the ELBO can be a meaningful lower bound on the log-likelihood: both are negative, but ELBO is lower. How much lower? The KL divergence from the conditional distribution.
I don’t see where you think the figure is indicating that it should be positive. The bottom of the diagram isn’t 0.

As for the second half of your question, see here: Should the likelihood function be increasing in every step of the EM algorithm?.
