Essential Math, Editors Pick, Mathematics, Data Science, Machine Learning.
distribution: You calculate the Shannon information of each probability of your distribution. You weight the Shannon information with the corresponding probability. You sum the weighted results. The entropy is the expectation of the information with respect to the probability distribution. Remember