Essential Math, Editors Pick, Mathematics, Data Science, Machine Learning.
of the entropy formula: log2 P(x). Figure 2: Illustration of the entropy as the weighted sum of the Shannon information. (Image by author) Each of these transformed probabilities is weighted by the corresponding raw probability. If an outcome occurs frequently, it will give more weight into the