Machine Learning, Statistics, Algorithms, Mathematics, Data Science.
introducing a measure known as the Kullback–Leibler divergence. Kullback-Leibler divergence It has 2 useful properties, that is it is always bigger or equal to 0 and it is only 0 if and only if P=Q. Thus, we can think of it as measuring the dissimilarity between P and Q. Recall how we derived the