Math, Statistics, Data Science, Information Theory, Decision Tree.
information entropy, a measure Shannon devised that can be interpreted as the level of ‘uncertainty’ or ‘surprise’ associated with the channel. We can see that the entropy has a maximum at 1 when p = ½, and minima at 0 for p = 0 and p = 1. More generally, given a random message M that can take n