Question: I'm watching a Machine Learning course offered by Alex Smola at <http://www.youtube.com/watch?v=OmJbGrQ93CA>

14 minutes in and im confused. At that part he starts to explain Linear Regression. I've watched other introduction to Machine Learning courses and also feel lost. I have difficulty understanding the equations even when their explained. I feel I've missed something. I understand there is no silver bullet and im not looking for one but for someone who is starting in this area and feels that the introduction is too difficult are there recommend resources I should be reading/watching to get up to speed in this area ?

I find other algorithms such as k-means and hierarchical clustering much easier to understand but maybe im not understanding at good enough level if im finding this course so difficult.

Answer: 
It's about the Poisson distribution. If $X$ is Poisson with mean $\lambda$, then the variance of $X$ is $\lambda$ also. This means that $$\frac{(X-\lambda)^2}{\lambda}$$ is a $z^2$ like entity. By the CLT, the Poisson tends to normality as the mean gets large, which is where the chi-squared comes in. Yes, it is an asymptotic test.

The degrees of freedom come from Cochran's theorem. Basically, Cochran explains how the Chi-squared is transformed (or remains unchanged) subject to a linear transformation in the $z^2$ scores. 

$$\sum\_i z\_i^2=Z' I Z$$

in matrix notation. If instead of computing the usual sum of squares, you compute $$Z' Q Z$$ for some matrix Q, then you still get a quantity with a a chi-squared distribution, but the degrees of freedom are now the rank of $Q$. There are more conditions on the matrix Q, but this is the gist of it.

If you play around with some matrix notation, you can express $$\sum\_i (z\_i-\bar{z})^2$$ as a quadratic form. Cochran assumes independence of the original normal variates, which is why the columns of your table of counts must be independent as well.
According to the textbook "Introductory Statistics with Randomization and Simulation", section 3.3.2 (textbook freely available at [OpenIntro](https://www.openintro.org/)), the $\chi^2$ test statistic is trying to accumulate the deviations of the observed from the expected. And the deviations are indeed expressed through the term 

$$Z\_i = \frac{O\_i - E\_i}{\sqrt{E\_i}}$$

which actually originates from $$\frac{O\_i - E\_i}{(Standard Error Of The Observed)}$$. 

The textbook goes on to say that the $(StandardErrorOfTheObserved)$ is better estimated by $\sqrt{E\_i}$, so the term becomes $Z\_i = \frac{O\_i - E\_i}{\sqrt{E\_i}}$. The textbook doesn't actually explain why this substitution is acceptable, and I'd also like to find out. 

Anyway, you could create a test statistic of the form

$$Z = |Z\_1| + |Z\_2| + |Z\_3| + ...$$

but it's better to square all the terms, because you get positive values immediately and the higher values stand out more after squaring. So you get the following:
$$\chi^2 = Z\_1^2 + Z\_2^2 + Z\_3^2 +...$$

But I don't know either why should this sum follow the $\chi^2$ distribution, or what's the connection to the definition of the $\chi^2$ distribution (sum of squares of standard normal independent variables).

**EDIT:** I'm still learning statistics, and I still don't think I understand the $\chi^2$ test properly. I hope others can enlighten me too.