What is the point of using a Bayesian prior? I do struggle with the most basic starting point of Bayesian statistics: why is using a prior useful?
It seems to me that if anything they hurt much more than help.
Moreover, Bayesians always say things like "the more evidence you get the less important a prior becomes". So why use them in the first place?
Especially if you have start with a prior that is really far-off, then you will be hurting your estimates.
To me a "Frequentist" approach seems much simpler and more straightforward.
I would like to discuss here an example that is very typical in introductory Bayesian course/explanations (e.g. this, this, or this)
Example 1 -- Are French people Rude?

Imagine that I am interested in estimating if French people are rude (in a binary way: Rude vs Non-Rude).
Imagine that the true parameter $\theta$ is 0.3, 30% of French are rude.
I have a random sample of 100 people with data on "rudeness".
A straightforward "Frequentist" approach would be: compute your confidence intervals on the sample and ... you done. We know what is the probability that the true $\theta$ lies within our CI, and we know that our sample average, on average, thanks to the Central Limit Theorem, will be close to the true parameter.
    true_theta = 0.3
    set.seed(111)

    # population
    X = rbinom(n = 10000, size = 1, prob = true_theta)

    # sample of 100
    x = sample(X, size = 100)
    p_hat = prop.table(table(x))[2]
    margin_error = 1.96 * sqrt( (p_hat*(1-p_hat)) / 100  
    ci_low = p_hat - margin_error
    ci_high = p_hat + margin_error
    data.frame(p_hat, ci_low, ci_high)

Now Bayesian will try to incorporate priors into this. Why?
We can imagine that most people will have beliefs that French are rude. (Using a Beta distribution) based on personal experience, someone would typically have a prior distribution like this: they know 10 French people and 7 are rude. So that is their prior.
The results are shown in the Figure above. On the bottom panel, I draw the Confidence Intervals around the sample mean.
It seems to me that the Frequentist approach will gives us a more precise and immediate answer to this question.
What am I missing here?
Example 2 -- Forecast Knock Out
Imagine I am interested in forecasting Knocked out (KO) in the Ultimate Fighting Championship (UFC).
My basic Frequentist approach would go like this.
Study the sport and see what variables play a role in predicting a fighter knocking another fighter (e.g. age of the fighters, winning streak, home advantage, …).
Then I will sample 30 UFC events and start build my model. I would use a simple logistic regression with maximum likelihood.
I don’t have prior per see, but I have a model of reality that I built on theoretical knowledge, like any scientists do, Bayesian or not.
My initial model would be that age and style of fighting (Muay Thai fighter vs BJJ) would be the most important factors predicting a KO.
I retrieve estimates from my logistic regression. Then I would cross-validate, take another sample and apply my initial model to the new data and see how it perform.
Let’s imagine that it doesn’t perform too well. I will then try to improve my model in studying more closely the sport. I find that accounting for an interaction between injury and age of the fighter is one of the most important predictors a KO. I re-run my regression with this and again cross validate, and now the model perform well.
Now I have a forecasting model telling me the probability of a KO and I can use it to bet money.
Furthermore, I know that certain special events, like a New Year's Eve event, will have more KOs, so I can adjust my model according to that fact.
Where would a prior, in the Bayesian sense, be useful here? Frequentists like all scientists use model to test reality and improve their models over time, but the difference is that they don’t put a formal initial probability on things. I still do not see why, you would want to do that.
    # code for the Figure
    theta_range <- seq(0, 1, by = 0.01)
    prior <- dbeta(x = theta_range, shape1 = 7, shape2 = 3)

    # observed success
    success_obs = table(x)[2]
    failure_obs = table(x)[1]

    # sampling distribution #
    likelihood <- dbinom(x = success_obs, size = 100, 
                           prob = theta_range) * 100

    # posterior 
    posterior_density <- likelihood * prior

   df <- data.frame(theta = theta_range,
                     likelihood = likelihood,
                     prior_dist = prior,
                     posterior_distribution = posterior_density)

    df_melt = melt(df, id.vars = 'theta')

    df_melt %>%
      ggplot(aes(x = theta, y = value, color = variable)) + 
      geom_line(size = 1.2) + 
      scale_x_continuous(breaks = seq(0, 1, by = 0.1)) + 
      geom_vline(xintercept = p_hat, size = 1.5) + 
      geom_vline(xintercept = ci_low, colour = 'gray', 
                                      size = 1.5) +
      geom_vline(xintercept = ci_high, colour = 'gray', 
                             size = 1.5) + theme_minimal() +
      ggtitle("Bayesian + Frequentist Confidence Intervals")


From whatever little I know about Bayesian method, the best use case is when you have limited pieces of evidence which by themselves are not enough to make a frequentist probability with reliability. But the little evidence you have can add to an informative prior probability to take you closer to the truth.
In the particular example you provided, I don't see a great use case for Bayesian application, but consider how you can solve this problem without using Bayesian interpretation - question from arbital

Suppose there's a bathtub full of coins:
Half the coins are "fair" and have a 50% probability of coming up Heads each time they are thrown.
A third of the coins are biased to produce Heads 25% of the time (Tails 75%).
The remaining sixth of the coins are biased to produce Heads 75% of the time.
You randomly draw a coin, flip it three times, and get the result HTH. What's the chance this is a fair coin?

Would it be doable if you throw away the prior information and see how you can answer the question maybe rephrased as "what is the probability of heads for a single toss of this coin which gave a HTH"?
