Confidence interval / p-value duality: don't they use different distributions? Idea:

*

*p-value is less than the level of significance if and only if the corresponding CI does not include the null value; and vica versa,

*the p-value is greater than the level of significance if and only if the corresponding CI does include the null value.

Is this idea always correct? Is it only correct for certain sampling distributions, like the normal, but not correct in general?
If the idea is correct in general, then why? The p-value is calculated using the distribution of the statistic conditional on H0, while the CI is calculated using the unconditional distribution of the statistic. These are two different distributions - how or why does this lead to the duality?
If the idea is not correct in general, could you provide counterexamples? Do people typically think that these counterexamples are a problem and somehow try to correct them, to make the idea still be true?

This is not correct in general.  I'll provide a simple example.
A popular test for a binomial proportion is derived from the central limit theorem.  If $\pi$ is the true risk for the outcome, then asymptotically,
$$ p \stackrel{d}{\approx} \mathcal{N}[\pi, \pi(1-\pi) / n] $$
where $p$ is our estimated risk and $n$ is our sample size.  The test is then found by standardizing $p$ from this distribution using either the estimated risk in the variance (Wald test) or the risk under the null (Score test).  The test statistic is
$$ Z=\frac{p-\pi_{0}}{\sqrt{\frac{p\left(1-p\right)}{n}}} $$
and associated confidence intervals are
$$ \left(\widehat{\pi}_{L}, \widehat{\pi}_{U}\right)=p \pm Z_{1-\alpha / 2} \sqrt{p(1-p) / n}$$
Your points in the bullets are true for this test and the associated confidence interval because the latter is derived from the former.  However, they fail in general as many different confidence intervals exist for the binomial $^{1.}$ all with close to nominal coverage and slightly different widths.  It could be the case that the test of proportions as shown above yields a p-value small enough to reject the null, but a confidence interval other than the one I've posted covers the null value.
We can demonstrate this with some R code.  I'll calculate the confidence intervals for a range and outcomes using a Wilson score interval and the asymptotic interval.  You will see they do not line up exactly, meaning some intervals cover some values while others don't, even considering the same data are used to create both.  Hence using some intervals we would reject the null so to speak, while using others would lead to a failure to reject the null.
library(binom)


n = 20
x  = seq(0, n, 2)

a = binom.wilson(x, n)
b = binom.asymp(x, n)


plot(a$upper, b$upper, xlab = "Wilson Upper Limit", ylab='Asymptotic Upper Limit', type = 'l', col='red')
abline(0, 1)



References


*

*Brown, Lawrence D., T. Tony Cai, and Anirban DasGupta. Confidence intervals for a binomial proportion and asymptotic expansions. The Annals of Statistics 30.1 (2002): 160-201.

