<p>If you have a finite set of possibilities (such as 0/1 for binary), the entropy is maximized, given that the probability of observing one or the other is 0.5, and nothing can increase it further</p>
<p>If you have a distribution that is close to the uniform, your entropy is high, but it's not maximized</p>

CC BY-SA 4.0
2024-01-26T08:37:52.593Z
2024-01-26T08:37:52.593Z
Alberto
<p>If the entropy of information reaches its maximum when all values of X are equally probable, indicating maximum uncertainty, do probabilities that don't vary much from the average contribute to greater entropy?</p>

CC BY-SA 4.0
2024-01-26T11:03:50.107Z
Can you please explain how this is related to AI? I know that this can be related to AI but it might not. So, please, provide more info about why you're asking this question.
nbro
<information-theory><entropy>
Do probabilities that don't vary much from the average contribute to greater entropy?
MAIKAO
# Do probabilities that don't vary much from the average contribute to greater entropy?

(Asked by: username_0 on 2024-01-26)

If the entropy of information reaches its maximum when all values of X are equally probable, indicating maximum uncertainty, do probabilities that don't vary much from the average contribute to greater entropy?

**Tags:** information-theory, entropy

> **Comments:**
> **username_1**: Can you please explain how this is related to AI? I know that this can be related to AI but it might not. So, please, provide more info about why you're asking this question.  

---

## 1 answers

### Answer by username_2 on 2024-01-26. Score: 0

If you have a finite set of possibilities (such as 0/1 for binary), the entropy is maximized, given that the probability of observing one or the other is 0.5, and nothing can increase it further

If you have a distribution that is close to the uniform, your entropy is high, but it's not maximized


---

