Testing if a sample follows a given distribution Suppose I have a single sample statistic $y$ (e.g., a scalar computed from the data collected after conducting an experiment) and would like to "test" if $y$ follows a given probability density function $p(x)$ (this is of course ill-posed - see below).

*

*How can I formally express this as a hypothesis testing problem? Also, is this unique? i.e., are there multiple ways of casting this task (described in words as "test if $y \sim p(x)$") as hypothesis testing problems?

*How do I do this for the special cases of (a) $p(x)= \mathcal{N}(\mu,\sigma^2)$ and (b) $p(x) = \chi^2_k$?


There are various ways in which to test whether a sample
might have been randomly chosen from a given distribution.
These might include checking to see if the sample mean is
consistent with the population, and similarly for variances, or other parameters.
I interpret your question to seek a test whether the
distribution of the sample (perhaps as expressed in its
empirical CDF) is consistent with the population distribution.
For example, suppose we happen to have a sample of size
$n = 500$ from an exponential distribution with mean $2$ and rate $1/2$ and so also standard deviation $2.$ A fictitious sample to these specifications is sampled and summarized in R below.
set.seed(2022)
x = rexp(500, 1/2)
summary(x);  sd(x)
     Min.   1st Qu.    Median      Mean   3rd Qu.      Max. 
 0.007349  0.569684  1.371701  1.987886  2.703088 12.677304 
[1] 1.923795  # sample SD

Not knowing how the sample was obtained, you might
have guessed that it could be from the population $\mathsf{Norm}(\mu=2, \sigma=2)$.
boxplot(x, horizontal=T, pch=19, col="skyblue2")


The shape of the boxplot already suggests
that this is not a normal sample.
One could use a Kolmogorov-Smirnov goodness-of-fit test
in R to test the null hypothesis that the population is
$\mathsf{Norm}(\mu=2, \sigma=2).$ This null hypothesis
is strongly rejected with P-value very nearly $0.$
ks.test(x, pnorm, 2, 2)

        One-sample Kolmogorov-Smirnov test

data:  x
D = 0.15955, p-value = 1.762e-11
alternative hypothesis: two-sided

More generally, if you had no idea of the population mean and
standard deviation, you could use one of several tests
to see if the sample is consistent with any normal distribution. One such test is the Shapiro-Wilk test, shown below using R; the null hypothesis of normality is strongly rejected.
shapiro.test(x)

        Shapiro-Wilk normality test

data:  x
W = 0.83826, p-value < 2.2e-16


Note: It is important to remember that the K-S test
is for a particular population distribution with all
parameters known. It is cheating to estimate the parameters
from the sample.
The K-S test does not reject the null hypothesis that the data x are from $\mathsf{Exp}(\mathrm{rate}= 1/2),$ but
strongly rejects that the population might have been
$\mathsf{Exp}(\mathrm{rate}= 1).$
ks.test(x, pexp, 1/2)

        One-sample Kolmogorov-Smirnov test

data:  x
D = 0.020534, p-value = 0.9843
alternative hypothesis: two-sided

ks.test(x, pexp, 1)

        One-sample Kolmogorov-Smirnov test

data:  x
D = 0.25929, p-value < 2.2e-16
alternative hypothesis: two-sided

The K-S test compares the empirical CDF (ECDF) of
the sample with the CDF of the population distribution. The test statistic $D$ is the maximum vertical distance between the two.
The plots below illustrate a good fit between ECDF and CDF at left, and a poor fit at right.

R code for figure:
par(mfrow=c(1,2))
 hdr1="Good fit to EXP(1/2)"
 plot(ecdf(x), main=hdr1)
  curve(pexp(x, 1/2), add=T, col="green3", lwd=3)
 hdr2="Bad fit to EXP(1)"
 plot(ecdf(x), main=hdr2)
  curve(pexp(x, 1), add=T, col="red", lwd=2)
par(mfrow=c(1,1))

