Statistical learning when observations are not iid As far as I am concerned, statistical/machine learning algorithms always suppose that data are independent and identically distributed ($iid$).
My question is: what can we do when this assumption is clearly unsatisfied? For instance, suppose that we have a data set whith repeated measurements on the same observations , so that both the cross-section and the time dimensions are important (what econometricians call a panel data set, or statisticians refer to as longitudinal data, which is distinct from a time series).
An example could be the following. In 2002, we collect the prices (henceforth $Y$) of 1000 houses in New York, together with a set of covariates (henceforth $X$). In 2005, we collect the same variables on the same houses. Similar happens in 2009 and 2012. Say I want to understand the relationship between $X$ and $Y$. Were the data $iid$, I could easily fit a random forest (or any other supervised algorithm, for what matters), thus estimating the conditional expectation of $Y$ given $X$. However, there is clearly some auto-correlation in my data. How can I handle this?

There are a few good answers here already but I thought it worth noting that the answer to this question can change drastically depending on how the iid assumption is violated. For example, if a univariate dataset is not iid, but is stationary, then many very simple estimation procedures, such as the sample mean, still converge to the appropriate limit.
However, if the iid assumption is violated because the data is non-stationary, then life is much more difficult. Note that the very common Machine Learning tradition of splitting the dataset into a training, test, and sometimes validation set, is invalid in the presence of non-stationarity. If this is the difficulty you face then your best bet is usually to try and find a transformation of the data that is close to stationary (or ergodicity) and work with that instead.
