Essential Math, Editors Pick, Mathematics, Data Science, Machine Learning.
distributions are identical. As you’ll see in 0.1.4, this property makes the cross-entropy a useful metric. Note also that the result is different according to the distribution you choose as a reference: H(P, Q) ≠ H(Q, P). Figure 5: Cross entropy can be used to compare the true distribution