Is there a Relationship Between Variance and Chi-Square? I am an MBA Student that is taking courses in Statistics.
Up until now, we had only encountered "Chi-Square" in the context of Contingency Tables. That is, how to find if the difference between different groups of subjects is statistically significant or non-significant (e.g. https://web.pdx.edu/~newsomj/uvclass/ho_chisq.pdf).
On the other hand, we have been learning about the concept of "Overdispersion" (related to variance). For example, we are learning about Regression Models for Discrete Count Data (e.g. Poisson GLM) in which an "Overdispersion Parameter" is required to be estimated, and this parameter will serve to "correct" the confidence intervals on the beta-coefficients of the regression model.
I would have thought that this "Overdispersion Parameter" would have been estimated through the Maximum Likelihood Equations - but our prof mentioned that the "Overdispersion Parameter" is estimated independently of the Maximum Likelihood Equations and is in fact calculated using a very similar formula as the "Chi-Square Statistic".
This makes me wonder:

*

*Is there some special relationship between "Overdispersion" (i.e. variance) and "Chi-Square" - and why isn't the Overdispersion Parameter estimated from the Maximum Likelihood Equations (as is done for the regression parameters)?


When we incorporate a dispersion parameter, then we are not working with real likelihood functions anymore.
Motivation, where does it come from
For several one parameter distributions (more precisely from the exponential family) it is possible to describe the derivative of the likelihood function as
$$\frac{\partial \log\mathcal{L}(x,\mu)} {\partial \mu}= \frac{x-\mu}{V(\mu)}$$
Where $V(\mu)$ is the variance of the distribution as function of the mean $\mu$.

*

*For instance for the Poisson distribution with mean $\mu$ and variance $V(\mu) = \mu$ the logarithm of the density is
$$\log[f(x,\mu)] = x \log(\mu) - \mu - \log(x!)$$
and the derivative with respect to location parameter $\mu$ is
$$\frac{\partial \log[f(x,\mu)]}{\partial \mu} = x/\mu - 1 = \frac{x-\mu}{\mu}$$
Including a dispersion parameter
This way to describe the likelihood function can be extended and we add a parameter
$$\frac{\partial \log\mathcal{L}(x,\mu,\phi)} {\partial \mu}= \frac{x-\mu}{\phi V(\mu)}$$
The effect of this parameter $\phi$ is that it changes the curvature of the likelihood function and this relates to more/less precise knowledge about the parameter $\mu$ depending on whether the likelihood is made sharper/blunt.
Finding the dispersion parameter (how it won't work with likelihood)
In this way of expressing the likelihood function, as the derivative with respect to $\mu$, the dispersion parameter does not play a role in the minimum.
So when you are minimizing this quasi likelihood function (quasi because it may not need to be the likelihood function of an actual existing distribution density) then the solution for $\mu$ is independent of $\phi$ which is only a scale factor.
We show the Gaussian distribution as an example to make more clear why this dispersion parameter disappear in the expression of the quasi likelihood function disappears

*

*For the Gaussian distribution with mean $\mu$ and variance $V(\mu) = \sigma^2$ (thus a constant function) the logarithm of the density is
$$\log[f(x,\mu, \sigma)] = -0.5 \log[\sigma^22 \pi] - 0.5 \frac{(\mu-x)^2}{\sigma^2}$$
and the derivative with respect to location parameter $\mu$ is
$$\frac{\partial \log[f(x,\mu, \sigma)]}{\partial \mu} = \frac{x-\mu}{\sigma^2} $$
This derivative is only expressing the second $ 0.5 \frac{(\mu-x)^2}{\sigma^2}$ term of the likelihood function. The part with $ -0.5 \log[\sigma^22 \pi]$, which depends on the dispersion $\sigma^2$ but not on the mean parameter $\mu$, is eliminated.
Finding the dispersion parameter (alternative)
So to find the dispersion parameter, one has to use a different trick. In Wedderburn 1974 this is done by using the method of moments.
The expectation of the variance is approximately:
$$E\left[\sum_{i=1}^n \frac{(x_i-\mu_i)^2}{V(\mu_i)} \right] \approx \phi({n-m})$$
(The approximation is based on assuming that $V(\mu_i)$ is approximately linear and that the distribution is $\chi^2$ distributed)
Then using the observed variance (with $\hat\mu_i$ in place of $\mu_i$) to estimate the expectation
$$\sum_{i=1}^n \frac{(x_i-\hat\mu_i)^2}{V(\hat\mu_i)} = \hat{E}\left[\sum_{i=1}^n \frac{(x_i-\mu_i)^2}{V(\mu_i)} \right]$$
gives
$$\hat\phi = \frac{1}{n-m} \sum_{i=1}^n \frac{(x_i-\hat\mu_i)^2}{V(\hat\mu_i)}$$
