text: Suppose that we have two multivariate normal distributions, with means  and with (non-singular) covariance matrices  If the two distributions have the same dimension, , then the relative entropy between the distributions is as follows:
  The logarithm in the last term must be taken to base e (mathematical constant)| since all terms apart from the last are base- logarithms of expressions that are either factors of the density function or otherwise arise naturally. The equation therefore gives a result measured in nats. Dividing the entire expression above by  yields the divergence in bits.
  In a numerical implementation, it is helpful to express the result in terms of the Cholesky decompositions  such that  and . Then with  and  solutions to the triangular linear systems , and ,
  A special case, and a common quantity in variational inference, is the relative entropy between a diagonal multivariate normal, and a standard normal distribution (with zero mean and unit variance):
  For two univariate normal distributions  and  the above simplifies to
  In the case of co-centered normal distributions with , this simplifies to: