Roughly how many bits are required on the average to describe to 3 digit accuracy the decay time (in years) of a radium atom if the half-life of radium is 80 years? Note that half-life is the median of the distribution.
Relevant Theorem: Differential entropy, also known as continuous entropy, is a concept in information theory that extends the idea of entropy from discrete random variables to continuous random variables. Entropy, in general, is a measure of the uncertainty or randomness associated with a random variable. In the context of information theory, it quantifies the average amount of information required to describe the outcome of a random variable.

For discrete random variables, entropy is well-defined using the Shannon entropy formula, which sums the product of the probability of each outcome and the logarithm of its reciprocal probability. However, for continuous random variables, the probability of any specific outcome is zero, making the Shannon entropy formula inapplicable.

Differential entropy addresses this issue by considering the probability density function (pdf) of a continuous random variable instead of the probabilities of individual outcomes. The differential entropy H(X) of a continuous random variable X with a probability density function f(x) is defined as:

H(X) = - ∫ f(x) * log(f(x)) dx

where the integral is taken over the entire range of the random variable X, and log is the logarithm base 2 (or any other base, depending on the desired unit of measurement for entropy).

Differential entropy can be interpreted as the average amount of information required to describe the outcome of a continuous random variable with a given probability density function. However, unlike the entropy of discrete random variables, differential entropy can be negative, which occurs when the probability density function is highly concentrated around certain values.

It is important to note that differential entropy is not a direct extension of discrete entropy, and some properties of discrete entropy do not hold for differential entropy. For example, differential entropy is not invariant under changes of variables or coordinate transformations, whereas discrete entropy is invariant under permutations of the outcomes.
19