Three questions about the article "Ditch p-values. Use Bootstrap confidence intervals instead" I am not a statistician by training and I was asked by students to explain them an article called "Ditch p-values. Use Bootstrap confidence intervals instead" . The author seems a prominent academic, however, I am confused about some of the material there. Please, ignore this post if it seems too long for you. I cut it to just 3 questions, I will infer other answers based on these.

Let’s take a simplified but revealing example: we want to determine Robert’s citizenship. Null hypothesis: H0, Robert is a US citizen. Alternative hypothesis: H1, he is not. Our data: we know that Robert is a US senator. There are 100 senators out of 330 million US citizens, so under the null hypothesis, the probability of our data (i.e., the p-value) is 100 / 300,000,000 ≈ 0.000000303. Per the rules of statistical significance, we can safely conclude that our null hypothesis is rejected and Robert is not a US citizen.

Am I right that this is not a p-value (which is the probability to see this or more extreme value of a test statistic)? Is it a correct procedure for a statistical testing? I have a gut feeling that it is a wrong situation to apply hypothesis testing, but I can not formally answer why.

P-values were invented at a time when all calculations had to be done by hand, and so they rely on simplifying statistical assumption. Broadly speaking, they assume that the phenomenon you’re observing obeys some regular statistical distribution

It seems to be wrong, but the question is: can we say that non-parametric tests also rely on some regular statistical distributions? Not only they have assumptions, but also, technically, their statistics also follow some distributions.

Let’s say that a business decision-maker is pondering two possible actions, A and B. Based on observed data, the probability of zero or negative benefits is:
0.08 for action A
0.001 for action B
Should the decision-maker pick action B based on these numbers? What if I told you that the corresponding 90%
confidence intervals are:
[-0.5m; 99.5m] for action A [0.1m; 0.2m] for action B Action B may
have a lower probability of leading to a zero or negative outcome, but
its expected value for the business is much lower, unless the business
is incredibly risk-averse.

Can we, based on confidence intervals, say, what is an expected value? Is in this situation a clear decision? I always thought that confidence intervals are not necessarily symmetric, but I started to doubt here.

The author of the article suffers from not understanding that hypothesis tests and confidence intervals serve different inferential purposes:

*

*The confidence interval (bootstrap or otherwise) serves to provide a plausible range of estimates for a target parameter.


*The hypothesis test serves to make a decision as to whether there is evidence or lack of evidence for a specific claim about a target parameter.
