Math, Statistics, Data Science, Information Theory, Decision Tree.
are technically only yes or no but this can be generalized). With this in mind, the entropy of a question becomes: The intuition here is that with each possible answer, we gain an amount of information — log p(xᵢ), meaning that if we receive an answer with a very low probability (i.e. we ask if the