Three questions about the article "Ditch p-values. Use Bootstrap confidence intervals instead" I am not a statistician by training and I was asked by students to explain them an article called "Ditch p-values. Use Bootstrap confidence intervals instead" . The author seems a prominent academic, however, I am confused about some of the material there. Please, ignore this post if it seems too long for you. I cut it to just 3 questions, I will infer other answers based on these.

Let’s take a simplified but revealing example: we want to determine Robert’s citizenship. Null hypothesis: H0, Robert is a US citizen. Alternative hypothesis: H1, he is not. Our data: we know that Robert is a US senator. There are 100 senators out of 330 million US citizens, so under the null hypothesis, the probability of our data (i.e., the p-value) is 100 / 300,000,000 ≈ 0.000000303. Per the rules of statistical significance, we can safely conclude that our null hypothesis is rejected and Robert is not a US citizen.

Am I right that this is not a p-value (which is the probability to see this or more extreme value of a test statistic)? Is it a correct procedure for a statistical testing? I have a gut feeling that it is a wrong situation to apply hypothesis testing, but I can not formally answer why.

P-values were invented at a time when all calculations had to be done by hand, and so they rely on simplifying statistical assumption. Broadly speaking, they assume that the phenomenon you’re observing obeys some regular statistical distribution

It seems to be wrong, but the question is: can we say that non-parametric tests also rely on some regular statistical distributions? Not only they have assumptions, but also, technically, their statistics also follow some distributions.

Let’s say that a business decision-maker is pondering two possible actions, A and B. Based on observed data, the probability of zero or negative benefits is:
0.08 for action A
0.001 for action B
Should the decision-maker pick action B based on these numbers? What if I told you that the corresponding 90%
confidence intervals are:
[-0.5m; 99.5m] for action A [0.1m; 0.2m] for action B Action B may
have a lower probability of leading to a zero or negative outcome, but
its expected value for the business is much lower, unless the business
is incredibly risk-averse.

Can we, based on confidence intervals, say, what is an expected value? Is in this situation a clear decision? I always thought that confidence intervals are not necessarily symmetric, but I started to doubt here.

"Am I right that this is not a p-value (which is the probability to see this or more extreme value of a test statistic)?" Good question! Yes, you're right, it's not a p-value. What's more the example is not a hypothesis test and it's not a significance test. Anyone who uses it as an argument to discard p-values or hypothesis tests is either confused or attempting to confuse.
The alleged p-value 100 / 300,000,000 is actually the likelihood of a person who is observed to be a US citizen being a senator! Not a p-value and, ironically, calculated on the assumption that the person is a citizen!
The example of US senators is badly adapted from Cohen's original (1994) that was intended to cast doubt on the utility of 'Fisher's disjunction' that underlies the evidential use of p-values.
On Cohen's argument against Fisher's disjunction:
[I've lifted this section from a paper (unpublished: rejected!) that I wrote a decade ago.]
A very highly cited paper that claims Fisher's disjunction is flawed is that of Cohen (1994), who illustrates his claim by drawing an analogy between Fisher's disjunction and this syllogism:

If a person is an American, then he is probably not a member of
Congress.
This person is a member of Congress.
Therefore, he is probably not an American.

As Cohen says, the last line of his syllogism about the American is false even though it would be true if the word `probably' were omitted from the first and last lines. However, he is incorrect in suggesting that it is directly analogous to Fisher's disjunction. As Hagen (1997) pointed out in a response published a few years after Cohen's paper, the null hypothesis in Fisher's disjunction refers to the population, whereas in Cohen's syllogism it refers to the sample.
Fisher's disjunction looks like this when put into the form of a syllogism:

*

*Extreme P-values from random samples are rare under the null
hypothesis.


*An extreme P-value has been observed.


*(Therefore, either a rare event has occurred or the null hypothesis
is false.)


*Therefore, the null hypothesis is probably false.
There is nothing wrong with that, although the line in parentheses is not logically necessary. When Cohen's syllogism is altered to refer to the population, it also is true:

*

*Members of Congress are rare in the population of Americans.


*This person is a member of Congress.


*(Therefore, either a rare event has occurred or this person is not a random sample from the population of Americans.)


*Therefore, this person is probably not a random sample from the population of Americans.
If a selected person turns out to be a member of Congress then an unusual event has occurred, or the person is a member of a non-American population in which members of Congress are more common, or the selection was not random. Assuming that all members of the American Congress are American there is no relevant non-American population from which the person might have been randomly selected, so the observation casts doubt on the random selection aspect. Cohen is incorrect in his assertion that the Fisher's disjunction lacks logical integrity.
(It is worth noting, parenthetically, that Cohen's paper contains many criticisms of null hypothesis testing that refer to problems arising from the use of what he describes as "mechanical dichotomous decisions around the sacred .05 criterion". He is correct in that, but the criticisms do not directly apply to P-values used as indices of evidence.)
Cohen, J. (1994). The earth is round (p <.05). American Psychologist, 49(12), 997.
Hagen, R. L. (1997). In praise of the null hypothesis statistical test. American Psychologist, 52(1), 15-24.
