Machine Learning, Entropy, Information Theory, Python.
From the construction of decision tree we have seen that the node by themselves may have higher impurity but knowing about other variable our entropy reduces so is the impurity. This is called information gain. So if we know the temperature is hot then entropy of me going for hike reduces.Let see