Gaussian Processes from Scratch (2019)
A very intuitive entry into gaussian processes comes from Chapter 12 of Statistical Rethinking by Richard McElreath:<p>He comes at it from the regression side and explains that GP&#x27;s basically occur when you have continuous variables in your regression problem like ages or income instead of individual units like countries or chimapanzee subjects. Here is a paragraph that sort of explains it<p>&gt; But what about continuous dimensions of variation like age or income or stature? Indi- viduals of the same age share some of the same exposures. They listened to some of the same music, heard about the same politicians, and experienced the same weather events. And individuals of similar ages also experienced some of these same exposures, but to a lesser extent than individuals of the same age. The covariation falls off as any two individuals be- come increasingly dissimilar in age or income or stature or any other dimension that indexes background similarity. It doesn’t make sense to estimate a unique varying intercept for all individuals of the same age, ignoring the fact that individuals of similar ages should have more similar intercepts.<p>The beauty of the author&#x27;s explanation is that Mixed slope and Intercept models are very <i>intuitive</i> and so are GP&#x27;s which are just their extension to the continuous random variables to model their covariances.<p>(BTW The author is explains &quot;regression&quot; of the kind used in Controlled Experiments in like social sciences or botanist and not really as an optimization problem in ML to reduce error; The coefficients are interpreted as effect sizes).