Essential Math, Editors Pick, Mathematics, Data Science, Machine Learning.
the Shannon information of Q(x) weighted according to the distribution of P(x). (Image by author) Figure 4 shows two different situations to illustrate the cross-entropy. On the left, you have two identical distributions P(x) (in blue) and Q(x) (in red). Their cross-entropy is equal to the entropy