Machine Learning, Artificial Intelligence, Data Science.
KL-divergence is always greater or equal to zero. Since entropy does not change regardless of how we model it, optimizing KL divergence w.r.t. model parameters is the same as optimizing cross-entropy. There are a few properties of the KL-divergence that need to be aware of. KL(p, q) ≥ 0 KL(p, p)=0,