Frequentists vs. Bayesians
Frequentists vs. Bayesians
[Caption above the first panel:] Did the sun just explode? (It's night, so we're not sure)
[Two Cueball-like guys stand on either side of a small table with a small black device on it. The device has white lines (ventilation) and two small antennas and a button on top. When the device speaks it uses in Westminster typeface. The Guy on the left, called Frequentist Statistician in the 2nd panel, points to the device. The guy on the right, called Bayesian Statistician in the 3rd panel, is just looking at the device. Above the spoken word from the device is a sound.] Frequentist Statistician: This neutrino detector measures whether the sun has gone nova. Bayesian Statistician: Then, it rolls two dice. If they both come up as six, it lies to us. Otherwise, it tells the truth. Frequentist Statistician: Let's try. Detector! Has the sun gone nova? Sound: Roll Device: YES.
[Two panels side by side are beneath the first panel. together they are broader than the top panel. Above each panel is a caption. In the left panel only the left statistician is shown with the device on the table. And in the right panel only the right statistician is shown with the device on the table. both are just looking at the device.] Frequentist Statistician: Frequentist Statistician: The probability of this result happening by chance is 1/36=0.027.  Since p<0.05, I conclude that the sun has exploded.
Bayesian Statistician: Bayesian Statistician: Bet you $50 it hasn't.

This comic is a joke about jumping to conclusions based on a simplistic understanding of probability. The " base rate fallacy " is a mistake where an unlikely explanation is dismissed, even though the alternative is even less likely. In the comic, a device tests for the (highly unlikely) event that the sun has exploded. A degree of random error is introduced, by rolling two dice and lying if the result is double sixes. Double sixes are unlikely (1 in 36, or about 3% likely), so the statistician on the left dismisses it. The statistician on the right has (we assume) correctly reasoned that the sun exploding is far more unlikely, and so is willing to stake money on his interpretation.
The labels given to the two statisticians, in their panels and in the comic's title, are not particularly fair or accurate, a fact which Randall has acknowledged: [1]
The " frequentist " statistician is (mis)applying the common standard of " p <0.05". In a scientific study, a result is presumed to provide strong evidence if, given that the null hypothesis , a default position that the observations are unrelated (in this case, that the sun has not gone nova), there is less than a 5% chance that the result was merely random. (The null hypothesis was also referenced in 892: Null Hypothesis .)
Since the likelihood of rolling double sixes is below this 5% threshold, the "frequentist" decides (by this rule of thumb) to accept the detector's output as correct. The " Bayesian " statistician has, instead, applied at least a small measure of probabilistic reasoning ( Bayesian inference ) to determine that the unlikeliness of the detector lying is greatly outweighed by the unlikeliness of the sun exploding. Therefore, he concludes that the sun has not exploded and the detector is lying.
A real statistician (frequentist or Bayesian) would probably demand a lower p -value before concluding that a test shows the Sun has exploded; physicists tend to use 5 sigma, or about 1 in 3.5 million, as the standard before declaring major results, like discovering new particles.  This would be equivalent to rolling between eight and nine dice and getting all sixes, although this is still not "very good" compared to the actual expected likelihood of the Sun spontaneously going nova, as discussed below.
The line, "Bet you $50 it hasn't", is a reference to the approach of a leading Bayesian scholar, Bruno de Finetti , who made extensive use of bets in his examples and thought experiments. See Coherence (philosophical gambling strategy) for more information on his work. In this case, however, the bet is also a joke because we would all be dead if the sun exploded.  If the Bayesian wins the bet, he gets money, and if he loses, they'll both be dead before money can be paid. This underlines the absurdity of the premise and emphasizes the need to consider context when examining probability.
It is also possible that the use of the sun is a reference to Laplace's Sunrise problem .
The title text refers to a classic series of logic puzzles known as Knights and Knaves , where there are two guards in front of two exit doors, one of which is real and the other leads to death. One guard is a liar and the other tells the truth. The visitor doesn't know which is which, and is allowed to ask one question to one guard. The solution is to ask either guard what the other one would say is the real exit, then choose the opposite. Two such guards were featured in the 1986 Jim Henson movie Labyrinth , hence the mention of "A LABYRINTH GUARD" here. A labyrinth was also mentioned in 246: Labyrinth Puzzle .
As mentioned, this is an instance of the base rate fallacy . If we treat the "truth or lie" setup as simply modelling an inaccurate test, then it is also specifically an illustration of the false positive paradox : A test that is rarely wrong, but which tests for an event that is even rarer, will be more often wrong than right when it says that the event has occurred.
The test, in this case, is a neutrino detector. It relies on the fact that neutrinos can pass through the earth, so a neutrino detector would detect neutrinos from the sun at all times, day and night. The detector is stated to give false results ("lie") 1/36th of the time.
There is no record of any star ever spontaneously exploding—they always show signs of deterioration long before their explosion—so the probability is near zero. For the sake of a number, though, consider that the sun's estimated lifespan is 10 billion years. Let's say the test is run every hour, twelve hours a day (at night time). This gives us a probability of the Sun exploding at one in 4.38×10 -13 . Assuming this detector is otherwise reliable, when the detector reports a solar explosion, there are two possibilities:
Clearly the sun exploding is not the most likely option. Indeed, Bayes' theorem can be used to find the probability that the Sun has exploded, given a result of "yes" and the prior probability given above:

[Caption above the first panel:] Did the sun just explode? (It's night, so we're not sure)
[Two Cueball-like guys stand on either side of a small table with a small black device on it. The device has white lines (ventilation) and two small antennas and a button on top. When the device speaks it uses in Westminster typeface. The Guy on the left, called Frequentist Statistician in the 2nd panel, points to the device. The guy on the right, called Bayesian Statistician in the 3rd panel, is just looking at the device. Above the spoken word from the device is a sound.] Frequentist Statistician: This neutrino detector measures whether the sun has gone nova. Bayesian Statistician: Then, it rolls two dice. If they both come up as six, it lies to us. Otherwise, it tells the truth. Frequentist Statistician: Let's try. Detector! Has the sun gone nova? Sound: Roll Device: YES.
[Two panels side by side are beneath the first panel. together they are broader than the top panel. Above each panel is a caption. In the left panel only the left statistician is shown with the device on the table. And in the right panel only the right statistician is shown with the device on the table. both are just looking at the device.] Frequentist Statistician: Frequentist Statistician: The probability of this result happening by chance is 1/36=0.027.  Since p<0.05, I conclude that the sun has exploded.
Bayesian Statistician: Bayesian Statistician: Bet you $50 it hasn't.
