Math, Statistics, Data Science, Information Theory, Decision Tree.
recursively always use the feature that maximizes the information gain. If we’re working with a dataset S and our feature is called X, the information gained on S by X, I(S,X), is calculated as: where H(S|X) is the conditional entropy of S given X. Intuitively, this is just the reduction in the