What is the difference between something being "true" and 'true with probability 1"? In the beginning of chapter 2 of Information Theory, Inference and Algorithms, the author says that he will refrain from being unnecessarily rigorous and provides the example of saying that something is "true" instead of saying more formally that said thing is "true with probability 1".
Is there a difference between something being "true" and 'true with probability 1" ?

A probability is just a function that satisfies a set of axioms, and maps subsets of the sample space to real numbers between $0$ and $1$. A widely used one is Kolmogorov axioms. Therefore, if you construct your own function that satisfies that set of axioms, no matter what it is, and you have a set which is mapped to $1$, then (the outcome represented by) that set is said to be true with probability $1$.
But when we say something is true, it is logically true. For example, if you flip a coin, it is true that you get either a head or a tail. It can be considered in the realm of “probability”, but it also has logical meaning.
