 Probability theory runs relatively smoothly in the finite realm, but puzzles emerge when infinities are afoot. There are multiple sources of infinitude, arising both in the mathematics and the interpretation of probability. We will firstly discuss more informally these sources, and then progress to more advanced issues.