What is the point of using a Bayesian prior? I do struggle with the most basic starting point of Bayesian statistics: why is using a prior useful?
It seems to me that if anything they hurt much more than help.
Moreover, Bayesians always say things like "the more evidence you get the less important a prior becomes". So why use them in the first place?
Especially if you have start with a prior that is really far-off, then you will be hurting your estimates.
To me a "Frequentist" approach seems much simpler and more straightforward.
I would like to discuss here an example that is very typical in introductory Bayesian course/explanations (e.g. this, this, or this)
Example 1 -- Are French people Rude?

Imagine that I am interested in estimating if French people are rude (in a binary way: Rude vs Non-Rude).
Imagine that the true parameter $\theta$ is 0.3, 30% of French are rude.
I have a random sample of 100 people with data on "rudeness".
A straightforward "Frequentist" approach would be: compute your confidence intervals on the sample and ... you done. We know what is the probability that the true $\theta$ lies within our CI, and we know that our sample average, on average, thanks to the Central Limit Theorem, will be close to the true parameter.
    true_theta = 0.3
    set.seed(111)

    # population
    X = rbinom(n = 10000, size = 1, prob = true_theta)

    # sample of 100
    x = sample(X, size = 100)
    p_hat = prop.table(table(x))[2]
    margin_error = 1.96 * sqrt( (p_hat*(1-p_hat)) / 100  
    ci_low = p_hat - margin_error
    ci_high = p_hat + margin_error
    data.frame(p_hat, ci_low, ci_high)

Now Bayesian will try to incorporate priors into this. Why?
We can imagine that most people will have beliefs that French are rude. (Using a Beta distribution) based on personal experience, someone would typically have a prior distribution like this: they know 10 French people and 7 are rude. So that is their prior.
The results are shown in the Figure above. On the bottom panel, I draw the Confidence Intervals around the sample mean.
It seems to me that the Frequentist approach will gives us a more precise and immediate answer to this question.
What am I missing here?
Example 2 -- Forecast Knock Out
Imagine I am interested in forecasting Knocked out (KO) in the Ultimate Fighting Championship (UFC).
My basic Frequentist approach would go like this.
Study the sport and see what variables play a role in predicting a fighter knocking another fighter (e.g. age of the fighters, winning streak, home advantage, …).
Then I will sample 30 UFC events and start build my model. I would use a simple logistic regression with maximum likelihood.
I don’t have prior per see, but I have a model of reality that I built on theoretical knowledge, like any scientists do, Bayesian or not.
My initial model would be that age and style of fighting (Muay Thai fighter vs BJJ) would be the most important factors predicting a KO.
I retrieve estimates from my logistic regression. Then I would cross-validate, take another sample and apply my initial model to the new data and see how it perform.
Let’s imagine that it doesn’t perform too well. I will then try to improve my model in studying more closely the sport. I find that accounting for an interaction between injury and age of the fighter is one of the most important predictors a KO. I re-run my regression with this and again cross validate, and now the model perform well.
Now I have a forecasting model telling me the probability of a KO and I can use it to bet money.
Furthermore, I know that certain special events, like a New Year's Eve event, will have more KOs, so I can adjust my model according to that fact.
Where would a prior, in the Bayesian sense, be useful here? Frequentists like all scientists use model to test reality and improve their models over time, but the difference is that they don’t put a formal initial probability on things. I still do not see why, you would want to do that.
    # code for the Figure
    theta_range <- seq(0, 1, by = 0.01)
    prior <- dbeta(x = theta_range, shape1 = 7, shape2 = 3)

    # observed success
    success_obs = table(x)[2]
    failure_obs = table(x)[1]

    # sampling distribution #
    likelihood <- dbinom(x = success_obs, size = 100, 
                           prob = theta_range) * 100

    # posterior 
    posterior_density <- likelihood * prior

   df <- data.frame(theta = theta_range,
                     likelihood = likelihood,
                     prior_dist = prior,
                     posterior_distribution = posterior_density)

    df_melt = melt(df, id.vars = 'theta')

    df_melt %>%
      ggplot(aes(x = theta, y = value, color = variable)) + 
      geom_line(size = 1.2) + 
      scale_x_continuous(breaks = seq(0, 1, by = 0.1)) + 
      geom_vline(xintercept = p_hat, size = 1.5) + 
      geom_vline(xintercept = ci_low, colour = 'gray', 
                                      size = 1.5) +
      geom_vline(xintercept = ci_high, colour = 'gray', 
                             size = 1.5) + theme_minimal() +
      ggtitle("Bayesian + Frequentist Confidence Intervals")


Here's a example of how to use Bayesian priors in a way that even "frequentists" agree is useful.
Let's say you want to estimate the know how well students at 100 different schools are doing in math, so you can identify schools that are doing particularly well or poorly. But you can only assess math knowledge through a test that not all students took. At most schools well over 100 students take the test but at some schools only a few take it due (assume nonresponse is random). Overall, the average score on test is 85% but obviously you are more interested in the average score at particular schools.
Now let's say that at school B only 5 students took the test and their average score was 50%. What are we to make of this?
A purely frequentist approach would to take the data at it's word and treat 50% as the best estimate available of the average score at that school, with some fairly large confidence intervals of course.
But that seems problematic. We know that overall the average test score across all students is an 85%. Given that we only have data from five students at this school, doesn't it seem MORE likely that the true value is actually somewhere closer to the 85% than 50%? The Bayesian approach to this problem would be to treat the overall mean as a "prior" and then update that prior with the additional data we got from the five students at this school. This is going to "shrink" our final estimate towards the mean by some amount. Since we only have five students at this school it's going to shrink it by quite a bit, since the data are weak. At a school with 150 respondents we would put more trust in the data and only shrink a little bit.
This approach is called "empirical Bayes estimation" and it's widely used in multilevel modeling, even by people who don't think of themselves as Bayesian, and more explicitly Bayesian versions of this approach (Google "Multilevel regression with post-stratification") are very common in political science when trying to get estimates of public opinion in small states
This is in fact what Fivethirtyeight.com does to predict elections in the US. To estimate the chance that a given candidate (say Trump) will win a state they look at polling at that state, but then they "shrink" the result of that polling towards a prior that they got from other data.
For example, let's say that before the 2020 campaign even starts you decide that, based on demographic trends, partisan affiliation and presidential approval numbers, Trump is only likely to get 30% of the vote in Vermont (this is our prior). Then someone does a poll of 100 people and finds that Trump is actually winning in Vermont with 51% of the vote. A frequentist would have to either put total trust in this result or ignore it completely. A Bayesian can do something more subtle: we use Bayes' rule to shrink this estimate towards our prior by some amount. In other words we don't actually believe that this poll means that Trump is actually ahead in Vermont, but we also no longer totally believe our old prior that Trump was only going to get 30%. Maybe now we think that we will win 40% of the vote. Then when we get even more new data, we update the new prior again.
Philosophical disagreements aside, this approach really works, which is why Fivethirthyeights's forecasts are so accurate:
https://projects.fivethirtyeight.com/checking-our-work/
