 Information as the negative log of the probability is the only mathematical function that exactly fulfills these constraints (Cover &amp; Thomas 2006). Shannon offers a theoretical framework in which binary strings can be interpreted as words in a (programming) language containing a certain amount of information (see  3.1 Languages).  The expression \(-\log p_x\) exactly gives the length of an optimal code for message x and as such formalizes the old intuition that codes are more efficient when frequent letters get shorter representations (see  3.2 Optimal codes).  Logarithms as a reduction of multiplication to addition (see  3.3 Numbers)  are a natural representation of extensive properties of systems and already as such had been used by physicists in the nineteenth century (see  3.4 Physics).