Empirical error in Kullback-Leibler KL divergence estimation In computing the Kullback-Leibler KL divergence $D(P\|Q)$ from an empirical data, it may happen that $Q(x)=0<P(x)$ at some sample point $x$ due to data error and $D(P\|Q)=\infty$. What are some good treatments of this singularity from estimation error?

Here is one possible solution. Use Jensen-Shannon divergence in place of Kullback-Leibler divergence. It resides in the interval $[0,1]$ in base $2$ logarithm.
