Machine Learning, Entropy, Information Theory, Python.
gini is default hyperparameter criterion for sklearn DecisionTreeClassifier, entropy is an alternative. Gini is outside the scope of this blog. Information gain Information gain is defined as how many bits on average would it save if both ends of line knew X. Information gain can be generalized as