How to apply the formula for Shannon entropy to a 4-sided die? I have come across calculating entropy, via the formula:
\begin{equation}
Entropy(p) = -\sum_{i=1} ^{N}p_i \log_2(p_i)
\end{equation}
Referring to this formula, how would I calculate the entropy of a fair 4 sided die for example?
I'm presuming 1/4 will be 'p', and 'i' will be the index?

I think it's worth mentioning that not only does the formula reduce to $\log N$ ($\log_2N$ if we're specifically talking about entropy measured in bits) in the case of $N$ different equally likely possibilities, but the formula for different probabilities is derived from the equally likely possibilities case, rather than the other way around. Shannon entropy is the number of bits it takes, on average, to specify a state. If all the states are equally likely, we can drop the "on average" part. Then we have that $S$ bits can specify $2^S$ states, or $\log_2 N$ bits are required to specify $N$ states. Specifying $4$ different states can clearly be done with $2$ bits; there are $4$ different $2$-digit binary numbers.
As for cases where the probabilities are different, suppose we have one state with $p=0.5$ and two states with $p = 0.25$. Then we can assign the first state the label $0$ and the other two states $10$ and $11$. Half the time we're using $1$ bit, and half the time we're using $2$ bits, so we have $1.5$ bits of entropy. It's more complicated when the probabilities aren't powers of $2$, but we can still give a similar calculation in the limit. That is, if we have a string of length $l$ consisting of independently random letters, and individual letters have an entropy of $S$, then we can on average represent the string with less than $S(l+1)$ bits, and the bits per letter goes to $S$ in the limit.
