<p>Please, take a look at <a href="https://arxiv.org/ftp/arxiv/papers/1405/1405.2061.pdf" rel="nofollow noreferrer">Understanding Shannon's Entropy metric for Information</a>. </p>

<p>The answer for the minus sign is in section 6. The probability logs are less than or equal to <span class="math-container">$0$</span>, so the minus sign guarantees that information (entropy) is always greater than or equal to <span class="math-container">$0$</span>.</p>

CC BY-SA 4.0
2017-09-27T13:32:13.553Z
2019-11-19T15:56:58.347Z
2019-11-19T15:56:58.347Z
xboard
<p>What is the intuition behind the following entropy formula used in the ID3 algorithm?</p>

<p><span class="math-container">$$
\text{info}(D) = -\sum_{i=1}^m p_i \log_2(p_i)
$$</span></p>

<machine-learning><math><decision-trees><id3-algorithm>
What is the intuition behind the entropy formula used in the ID3 algorithm?
DRV
# What is the intuition behind the entropy formula used in the ID3 algorithm?

(Asked by: username_0 on 2017-09-27)

What is the intuition behind the following entropy formula used in the ID3 algorithm?

$$
\text{info}(D) = -\sum\_{i=1}^m p\_i \log\_2(p\_i)
$$

**Tags:** machine-learning, math, decision-trees, id3-algorithm


---

## 1 answers

### Answer by username_1 on 2017-09-27. Score: 2 | Accepted answer

Please, take a look at [Understanding Shannon's Entropy metric for Information](https://arxiv.org/ftp/arxiv/papers/1405/1405.2061.pdf).

The answer for the minus sign is in section 6. The probability logs are less than or equal to $0$, so the minus sign guarantees that information (entropy) is always greater than or equal to $0$.


---

