Philosophy, Information Theory, Energy, Thermodynamics.
Option 1. Therefore, Option 2 communicates more information because it reduces uncertainty to a greater degree. So if information is resolved uncertainty, entropy must be the uncertainty needing resolving. The rarity of an event, or its “surprisal”, determines its information content (if you’ve