Database, Decision Tree, Random Forest, Entropy, Gini Index.
={x1, x2, …, xn}, the entropy of X is defined as follows: where b = 10 and P(xi) is the probability of xi in X. Note that if P(xi) = 0, then we consider P(xi) log P(xi) = 0. Also, the closer H(x) is to 0, the better, since less entropy implies more information gain). Example Let’s look at the