<p>It is not possible to have a negative IG, your outcome is negative because of a computational mistake: initially you have 69 positive instances, and after creating 2 children the sum drops to 56+11=67. Set is therefore less pure -> IG is negative.</p>

CC BY-SA 4.0
2018-10-19T10:32:57.197Z
2018-10-19T10:32:57.197Z
user19149
<p>I'm creating a Decision Tree and at the very root level itself, I'm getting <strong>negative Information Gain</strong>. </p>

<p>As per my knowledge, Information Gain is always <code>&gt; 0</code>....</p>

<p>Any explanation....please....</p>

<p>Please look at the node below.....</p>

<pre><code>    [69+,42-]
     /   \        
    /     \                       
[56+,33-]  [11+,11-]

IG = H([69+, 42-]) - H([56+,33-],[11+,11-]) 

   = {(-69/111 * lg(69/111) -42/111 * lg(42/111)) 
      - 89/111 * ( -56/89*lg(56/89) -33/89*lg(33/89)) 
      - 22/111 * ( -11/22*lg(11/22) -11/22*lg(11/22))}

   = -0.004
</code></pre>

<p>It turns out that for every feature, the <code>IG &lt; 0</code>.</p>

<p>What should I do to decide the feature at the root node?</p>

<p>Thanks</p>

<machine-learning><decision-theory>
Getting Negative Information Gain (using Entropy as a measure of disorder)
imflash217
# Getting Negative Information Gain (using Entropy as a measure of disorder)

(Asked by: username_0 on 2017-12-18)

I'm creating a Decision Tree and at the very root level itself, I'm getting negative Information Gain. 
As per my knowledge, Information Gain is always > 0....
Any explanation....please....
Please look at the node below.....
    [69+,42-]
     /   \        
    /     \                       
[56+,33-]  [11+,11-]

IG = H([69+, 42-]) - H([56+,33-],[11+,11-]) 

   = {(-69/111 * lg(69/111) -42/111 * lg(42/111)) 
      - 89/111 * ( -56/89*lg(56/89) -33/89*lg(33/89)) 
      - 22/111 * ( -11/22*lg(11/22) -11/22*lg(11/22))}

   = -0.004

It turns out that for every feature, the IG < 0.
What should I do to decide the feature at the root node?
Thanks

**Tags:** machine-learning, decision-theory


---

## 1 answers

### Answer by username_1 on 2018-10-19. Score: 2

It is not possible to have a negative IG, your outcome is negative because of a computational mistake: initially you have 69 positive instances, and after creating 2 children the sum drops to 56+11=67. Set is therefore less pure -> IG is negative.


---

