What is the point of using a Bayesian prior? I do struggle with the most basic starting point of Bayesian statistics: why is using a prior useful?
It seems to me that if anything they hurt much more than help.
Moreover, Bayesians always say things like "the more evidence you get the less important a prior becomes". So why use them in the first place?
Especially if you have start with a prior that is really far-off, then you will be hurting your estimates.
To me a "Frequentist" approach seems much simpler and more straightforward.
I would like to discuss here an example that is very typical in introductory Bayesian course/explanations (e.g. this, this, or this)
Example 1 -- Are French people Rude?

Imagine that I am interested in estimating if French people are rude (in a binary way: Rude vs Non-Rude).
Imagine that the true parameter $\theta$ is 0.3, 30% of French are rude.
I have a random sample of 100 people with data on "rudeness".
A straightforward "Frequentist" approach would be: compute your confidence intervals on the sample and ... you done. We know what is the probability that the true $\theta$ lies within our CI, and we know that our sample average, on average, thanks to the Central Limit Theorem, will be close to the true parameter.
    true_theta = 0.3
    set.seed(111)

    # population
    X = rbinom(n = 10000, size = 1, prob = true_theta)

    # sample of 100
    x = sample(X, size = 100)
    p_hat = prop.table(table(x))[2]
    margin_error = 1.96 * sqrt( (p_hat*(1-p_hat)) / 100  
    ci_low = p_hat - margin_error
    ci_high = p_hat + margin_error
    data.frame(p_hat, ci_low, ci_high)

Now Bayesian will try to incorporate priors into this. Why?
We can imagine that most people will have beliefs that French are rude. (Using a Beta distribution) based on personal experience, someone would typically have a prior distribution like this: they know 10 French people and 7 are rude. So that is their prior.
The results are shown in the Figure above. On the bottom panel, I draw the Confidence Intervals around the sample mean.
It seems to me that the Frequentist approach will gives us a more precise and immediate answer to this question.
What am I missing here?
Example 2 -- Forecast Knock Out
Imagine I am interested in forecasting Knocked out (KO) in the Ultimate Fighting Championship (UFC).
My basic Frequentist approach would go like this.
Study the sport and see what variables play a role in predicting a fighter knocking another fighter (e.g. age of the fighters, winning streak, home advantage, …).
Then I will sample 30 UFC events and start build my model. I would use a simple logistic regression with maximum likelihood.
I don’t have prior per see, but I have a model of reality that I built on theoretical knowledge, like any scientists do, Bayesian or not.
My initial model would be that age and style of fighting (Muay Thai fighter vs BJJ) would be the most important factors predicting a KO.
I retrieve estimates from my logistic regression. Then I would cross-validate, take another sample and apply my initial model to the new data and see how it perform.
Let’s imagine that it doesn’t perform too well. I will then try to improve my model in studying more closely the sport. I find that accounting for an interaction between injury and age of the fighter is one of the most important predictors a KO. I re-run my regression with this and again cross validate, and now the model perform well.
Now I have a forecasting model telling me the probability of a KO and I can use it to bet money.
Furthermore, I know that certain special events, like a New Year's Eve event, will have more KOs, so I can adjust my model according to that fact.
Where would a prior, in the Bayesian sense, be useful here? Frequentists like all scientists use model to test reality and improve their models over time, but the difference is that they don’t put a formal initial probability on things. I still do not see why, you would want to do that.
    # code for the Figure
    theta_range <- seq(0, 1, by = 0.01)
    prior <- dbeta(x = theta_range, shape1 = 7, shape2 = 3)

    # observed success
    success_obs = table(x)[2]
    failure_obs = table(x)[1]

    # sampling distribution #
    likelihood <- dbinom(x = success_obs, size = 100, 
                           prob = theta_range) * 100

    # posterior 
    posterior_density <- likelihood * prior

   df <- data.frame(theta = theta_range,
                     likelihood = likelihood,
                     prior_dist = prior,
                     posterior_distribution = posterior_density)

    df_melt = melt(df, id.vars = 'theta')

    df_melt %>%
      ggplot(aes(x = theta, y = value, color = variable)) + 
      geom_line(size = 1.2) + 
      scale_x_continuous(breaks = seq(0, 1, by = 0.1)) + 
      geom_vline(xintercept = p_hat, size = 1.5) + 
      geom_vline(xintercept = ci_low, colour = 'gray', 
                                      size = 1.5) +
      geom_vline(xintercept = ci_high, colour = 'gray', 
                             size = 1.5) + theme_minimal() +
      ggtitle("Bayesian + Frequentist Confidence Intervals")


A prior is useful for incorporating historical information, analogous to a fixed effect meta-analysis.  Because Bayesians define parameters as random variables and probability as the belief of the experimenter they may feel much freer to incorporate information from other studies or their personal beliefs.  In contrast, because frequentists are concerned with the performance of their testing procedure they may scrutinize the compatibility of studies.  The frequentist needs to feel comfortable assuming the unknown fixed true quantity being investigated in each study is identical (even if the subject-level observations are not necessarily exchangeable).  This may be a big assumption to make, analgous to assuming an informative Bayesian prior.  Using a purely subjective Bayesian prior is analogous to performing a meta-analysis with hypothetical experimental data.
I will argue against using a Bayesian prior and in favor of a frequentist meta-analysis, beginning with an example taken from Decision Making in Drug Development via Confidence Distributions (Johnson 2021).  The primary reason not to use a Bayesian prior is that a subjective interpretation of probability as a measure of belief is unfalsifiable (1)(2).
A confidence distribution $H(\theta,\boldsymbol{x})$ as a function of the hypothesis and observed data has the appearance of a cdf on the parameter space and depicts p-values and confidence intervals of all levels based on a particular testing procedure.  This same information can be displayed as a confidence density and a confidence curve.
The figure below depicts a meta-analysis using confidence distributions for a binomial proportion $\theta$. Density (a) represents an informative prior distribution based on historical data and a vague conjugate prior with an estimate of 0.90 and a sample size of n = 50. This same information is depicted in (b) as a confidence density resulting from a likelihood ratio test. A similar confidence density can be produced by inverting a Wald test with a logit link. The posterior based on the current data binomial likelihood and a vague conjugate prior is shown in (c) with an estimate of 0.87 resulting from n = 30. This same information can be represented as a likelihood ratio confidence density, (d). Using Bayes theorem, (a) and (c) combine to form (e). Multipliying the historical and current likelihoods and inverting a likelihood ratio test forms (f). This multiplication of independent likelihoods is precisely what Bayes theorem accomplishes (plus normalization), without the inversion of a hypothesis test.

The confidence densities above can be interpreted under a Neyman-Pearson framework using a pre-specified null hypothesis and type I error rate.  they can also be interpreted under a Fisherian framework of evidential p-values to compare the plausibility of multiple hypotheses, not necessarily pre-specified.  Many Bayesians will use a Dutch book argument in support of posterior probability.  Here is an example where the long-run characteristics of a likelihood ratio test are used to form a Dutch book against two different Bayesian posteriors. (3)
