Machine Learning, Decision Tree, Information Gain, Entropy.
is an equally divided it has entropy of one[1]. n-class Entropy -> E(S) = ∑ -(pᵢ*log₂pᵢ) 2-class Entropy: (S) =-(p₁ * log₂p₁ + p₂ * log₂p₂) Ex-1: 9 samples in the Class 1 and 5 samples in the Class 2 p₁ = 9/(9+5) =9/14 (probability of a sample in the same training class with class 1) p₂ = 5/14