<p>To further elaborate on the answer of @Michael Gmeiner, the difference in the estimate of the variance between the model that takes into account clustering and the model without cluster is usually expressed with what is called, Moulton factor. See also  <a href="https://doi.org/10.1016/0304-4076(86)90021-7" rel="nofollow noreferrer">Moulton (1986), Random group effects and the precission of regression estimates.</a></p>
<p>Consider the basic case with <span class="math-container">$G$</span> groups (clusters) indexed by <span class="math-container">$g$</span>. Assume that there are in total <span class="math-container">$N$</span> observations with <span class="math-container">$N_g$</span> observations in group <span class="math-container">$g$</span>. Individual observations are indexed by <span class="math-container">$i$</span>. Consider the following simple regression with a single covariate:
<span class="math-container">$$
y_{i,g} = \alpha + \beta x_{i,g} + \varepsilon_{i,g}.
$$</span>
We would like to see how the estimated variance of the OLS estimate of <span class="math-container">$\beta$</span> differs when we take intra group correlation (clustering) into account and when not.</p>
<p>To model intra-group correlation, take the case where for <span class="math-container">$i \ne j$</span> and <span class="math-container">$g \ne g'$</span>,
<span class="math-container">$$
\begin{align*}
&amp;\mathbb{E}(\varepsilon_{i,g}) = 0,\\
&amp;\mathbb{E}(\varepsilon_{i,g}^2) = \sigma^2,\\
&amp;\mathbb{E}(\varepsilon_{i,g}\varepsilon_{j,g'}) = 0,\\
&amp;\mathbb{E}(\varepsilon_{i,g}\varepsilon_{j,g}) = \sigma^2 \rho.
\end{align*}
$$</span>
So there is no correlation of the residuals across groups <span class="math-container">$(g \ne g')$</span> but there is correlation of the residuals within each group, denoted by <span class="math-container">$\rho$</span>. We assume that <span class="math-container">$\sigma^2$</span> and <span class="math-container">$\rho$</span> is the same for all groups. Given the residuals <span class="math-container">$\hat \varepsilon_{i,g}$</span> of the OLS, we can estimate these using:
<span class="math-container">$$
\begin{align*}
&amp;\hat \sigma^2 = \sum_{g} \sum_{i \in g} \hat \varepsilon_{i,g}^2,\\
&amp;\hat \rho = \frac{\sum_{g} \sum_{i \in g} \sum_{j \in g, j \ne i} \varepsilon_{i,g} \varepsilon_{j,g}}{\hat \sigma^2 \sum_g N_g(N_g - 1)}
\end{align*}
$$</span>
The OLS estimate <span class="math-container">$\hat \beta$</span> of <span class="math-container">$\beta$</span>, is given as follows:
<span class="math-container">$$
\hat \beta = \frac{\sum_{g} \sum_{i \in g} (y_{i,g}- \overline{y})(x_{i,g}-\overline{x})}{\sum_g \sum_{i} (x_{i,g} - \overline{x})^2}
$$</span>
Here <span class="math-container">$\overline{x}$</span> and <span class="math-container">$\overline{y}$</span> are the sample averages of <span class="math-container">$x$</span> and <span class="math-container">$y$</span>. Substituting <span class="math-container">$y_{i,g} = \alpha + \beta x_{i,g} + \varepsilon_{i,g}$</span> gives:
<span class="math-container">$$
\hat \beta - \beta = \frac{\sum_g \sum_{i \in g}(\varepsilon_{i,g})(x_{i,g} - \overline{x})}{\sum_g\sum_i (x_{i,g} - \overline{x})^2}
$$</span>
Notice that:
<span class="math-container">$$
var(\hat \beta) = \mathbb{E}(\hat \beta -\beta)^2 = \mathbb{E}\left(\frac{\left(\sum_{g} \sum_{i \in g} \varepsilon_{i,g} (x_{i,g}- \overline{x})\right)^2}{\left(\sum_g \sum_{i \in g} (x_{i,g} - \overline{x})^2\right)^2}\right) \tag{1}
$$</span>
We would like to estimate this expression. Define the inter-group (sample) correlation of <span class="math-container">$x$</span>:
<span class="math-container">$$
\hat \rho_x = \frac{\sum_g \sum_{i \in g} \sum_{j \in g, i \ne j} (x_{i,g} - \overline{x})(x_{j,g} - \overline{x})}{\sum_g N_g (N_g - 1)} \frac{1}{\frac{1}{N}\sum_g \sum_i (x_{i,g} - \overline{x})^2 },\\
\to \sum_g \sum_{i \ne j} (x_{i,g} - \overline{x})(x_{j,g} - \overline{x}) = \hat \rho_x \sum_g  \sum_i (x_{i,g} - \overline{x})^2 \frac{1}{N} \sum_g N_g(N_g -1)
$$</span></p>
<p>Which is the sample estimator of <span class="math-container">$corr(x_{i,g}, x_{j,g})$</span>. The numerator of the expression <span class="math-container">$(1)$</span> can then be estimated by:
<span class="math-container">$$
\begin{align*}
&amp;\sum_g \sum_{g'} \sum_{i \in g} \sum_{j \in g'} \hat \varepsilon_{i,g} \hat \varepsilon_{j,g'} (x_{i,g}-\overline{x}) (x_{j,g'}-\overline{x})\\
&amp;\approx \hat \sigma^2 \sum_g \sum_{i \in g} (x_{i,g}- \overline{x})^2 + \hat \sigma^2 \hat \rho \sum_{g} \sum_{i \in g} \sum_{j \in g, i \ne j} (x_{i,g}-\overline{x})(x_{j,g} - \overline{x}),\\
&amp;= \hat \sigma^2 \sum_g \sum_{i \in g} (x_{i,g} - \overline{x})^2 + \hat \sigma^2 \hat \rho \hat \rho_x \frac{1}{N} \sum_{i}(x_{i,g} - \overline{x})^2 \sum_g N_g(N_g-1),\\
&amp;=\hat \sigma^2 \sum_g \sum_i (x_{i,g} - \overline{x})^2\left[1 + \left(\frac{1}{N} \sum_g N_g^2 - 1\right)\hat \rho \hat \rho_x \right]
\end{align*}
$$</span>
Next, observe that:
<span class="math-container">$$
\begin{align*}
var(N_g) &amp;= \frac{1}{G}\sum_g N_g^2 - \frac{N^2}{G^2},\\
&amp;\to \frac{1}{N}\sum_g N_g^2 = \frac{var(N_g)}{\overline{m}} + \overline{m}
\end{align*}
$$</span>
where <span class="math-container">$\overline{m}$</span> is the average group size. Then we have that:
<span class="math-container">$$
\mathbb{E}(\hat \beta - \beta)^2 \approx \frac{\hat \sigma^2 \sum_g \sum_{i \in g} (x_{i,g} - \overline{x})^2 \left(1 + \hat \rho\hat \rho_x \left(\frac{var(N_g)}{\overline{m}} + \overline{m} -1\right)  \right)}{\left(\sum_g \sum_i (x_i - \overline{x})^2\right)^2} \tag{2}
$$</span>
If we would disregard the clustering, i.e. set <span class="math-container">$\hat \rho = 0$</span>, we would obtain the estimator:
<span class="math-container">$$
\mathbb{E}(\hat \beta - \beta)^2 \approx \frac{\hat \sigma^2 \sum_g \sum_i (x_{i,g} - \overline{x})^2}{\left(\sum_g \sum_i (x_{i,g} - \overline{x})^2\right)^2} \tag{3}
$$</span>
The ratio of these two estimators <span class="math-container">$(2)$</span> and <span class="math-container">$(3)$</span> is then:
<span class="math-container">$$
\tau = 1 + \left[\frac{var(N_g)}{\overline{m}} + \overline{m} - 1\right]\hat \rho \hat \rho_x
$$</span>
This is called the Moulton factor. So if <span class="math-container">$V(\hat \beta)$</span> is the estimated variance for the clustered model and <span class="math-container">$\hat V(\hat \beta)$</span> is the one for the model that disregards clustering, we have:
<span class="math-container">$$
V(\hat \beta) = \tau\,\, \hat V(\hat \beta).
$$</span>
We see that <span class="math-container">$\tau &gt; 1$</span> if <span class="math-container">$\hat \rho, \hat \rho_x &gt; 0$</span> which will be the case if there is positive intra-group correlation. If the intra-group correlation is negative, but <span class="math-container">$\rho_x &gt; 0$</span>, we have that <span class="math-container">$\tau &lt; 1$</span>. so clustering will decrease the variance as menioned by  @Michael Gmeiner</p>
<p>Now, consider the special case where <span class="math-container">$x_{i,g}$</span> only varies over groups. So <span class="math-container">$x_{i,g} = x_g$</span> for all <span class="math-container">$i \in g$</span> and where all groups have the same size. Then we have that <span class="math-container">$\hat \rho_x = 1$</span>, <span class="math-container">$var(N_g) = 0$</span> and <span class="math-container">$\overline{m} = N_g$</span>. Then:
<span class="math-container">$$
\tau = 1 + (N_g - 1)\hat \rho.
$$</span>
So the increase in the estimated variance will be increasing in the intra-group correlation and the number of observations in each group. In general we can expect that <span class="math-container">$\hat \rho &gt; 0$</span> so clustered variances will be higher than the standard estimate that does not take into account clustering.</p>

CC BY-SA 4.0
2021-07-12T08:19:35.133Z
2021-07-12T08:19:35.133Z
tdm
<p>It can either increase or decrease when clustering, it depends on the covariance structure of the error terms. It is more frequent to observe standard errors increase with clustering.</p>

CC BY-SA 4.0
2021-07-09T23:28:43.433Z
is there any intuitive example regarding "***standard errors decreases with clustering***"
Phil Nguyen
CC BY-SA 4.0
2021-07-10T11:20:04.837Z
If individuals in a group have negatively correlated errors, then if each get a positive shock to their error term, the negative correlated will attenuate the effects of that shock. When we have an estimator of variance that accounts for this correlation, we will have a smaller variance.
Michael Gmeiner
CC BY-SA 4.0
2021-07-08T11:03:26.200Z
2021-07-08T11:03:26.200Z
Michael Gmeiner
<p>In panel data, when adding covariates, the standard error normally decreased, I am wondering whether standard error when clustering higher or lower comparing to without clustering.</p>

<econometrics>
How clustering affects standard error?
Phil Nguyen
# How clustering affects standard error?

(Asked by: username_0 on 2021-07-08)

In panel data, when adding covariates, the standard error normally decreased, I am wondering whether standard error when clustering higher or lower comparing to without clustering.

**Tags:** econometrics


---

## 2 answers

### Answer by username_2 on 2021-07-08. Score: 7 | Accepted answer

It can either increase or decrease when clustering, it depends on the covariance structure of the error terms. It is more frequent to observe standard errors increase with clustering.

> **Comments:**
> **username_0**: is there any intuitive example regarding "***standard errors decreases with clustering***"  
> **username_2**: If individuals in a group have negatively correlated errors, then if each get a positive shock to their error term, the negative correlated will attenuate the effects of that shock. When we have an estimator of variance that accounts for this correlation, we will have a smaller variance.  

---

### Answer by username_1 on 2021-07-12. Score: 2

To further elaborate on the answer of @username_2, the difference in the estimate of the variance between the model that takes into account clustering and the model without cluster is usually expressed with what is called, Moulton factor. See also [Moulton (1986), Random group effects and the precission of regression estimates.](https://doi.org/10.1016/0304-4076(86)90021-7)

Consider the basic case with $G$ groups (clusters) indexed by $g$. Assume that there are in total $N$ observations with $N\_g$ observations in group $g$. Individual observations are indexed by $i$. Consider the following simple regression with a single covariate:
$$
y\_{i,g} = \alpha + \beta x\_{i,g} + \varepsilon\_{i,g}.
$$
We would like to see how the estimated variance of the OLS estimate of $\beta$ differs when we take intra group correlation (clustering) into account and when not.

To model intra-group correlation, take the case where for $i \ne j$ and $g \ne g'$,
$$
\begin{align\*}
&\mathbb{E}(\varepsilon\_{i,g}) = 0,\\
&\mathbb{E}(\varepsilon\_{i,g}^2) = \sigma^2,\\
&\mathbb{E}(\varepsilon\_{i,g}\varepsilon\_{j,g'}) = 0,\\
&\mathbb{E}(\varepsilon\_{i,g}\varepsilon\_{j,g}) = \sigma^2 \rho.
\end{align\*}
$$
So there is no correlation of the residuals across groups $(g \ne g')$ but there is correlation of the residuals within each group, denoted by $\rho$. We assume that $\sigma^2$ and $\rho$ is the same for all groups. Given the residuals $\hat \varepsilon\_{i,g}$ of the OLS, we can estimate these using:
$$
\begin{align\*}
&\hat \sigma^2 = \sum\_{g} \sum\_{i \in g} \hat \varepsilon\_{i,g}^2,\\
&\hat \rho = \frac{\sum\_{g} \sum\_{i \in g} \sum\_{j \in g, j \ne i} \varepsilon\_{i,g} \varepsilon\_{j,g}}{\hat \sigma^2 \sum\_g N\_g(N\_g - 1)}
\end{align\*}
$$
The OLS estimate $\hat \beta$ of $\beta$, is given as follows:
$$
\hat \beta = \frac{\sum\_{g} \sum\_{i \in g} (y\_{i,g}- \overline{y})(x\_{i,g}-\overline{x})}{\sum\_g \sum\_{i} (x\_{i,g} - \overline{x})^2}
$$
Here $\overline{x}$ and $\overline{y}$ are the sample averages of $x$ and $y$. Substituting $y\_{i,g} = \alpha + \beta x\_{i,g} + \varepsilon\_{i,g}$ gives:
$$
\hat \beta - \beta = \frac{\sum\_g \sum\_{i \in g}(\varepsilon\_{i,g})(x\_{i,g} - \overline{x})}{\sum\_g\sum\_i (x\_{i,g} - \overline{x})^2}
$$
Notice that:
$$
var(\hat \beta) = \mathbb{E}(\hat \beta -\beta)^2 = \mathbb{E}\left(\frac{\left(\sum\_{g} \sum\_{i \in g} \varepsilon\_{i,g} (x\_{i,g}- \overline{x})\right)^2}{\left(\sum\_g \sum\_{i \in g} (x\_{i,g} - \overline{x})^2\right)^2}\right) \tag{1}
$$
We would like to estimate this expression. Define the inter-group (sample) correlation of $x$:
$$
\hat \rho\_x = \frac{\sum\_g \sum\_{i \in g} \sum\_{j \in g, i \ne j} (x\_{i,g} - \overline{x})(x\_{j,g} - \overline{x})}{\sum\_g N\_g (N\_g - 1)} \frac{1}{\frac{1}{N}\sum\_g \sum\_i (x\_{i,g} - \overline{x})^2 },\\
\to \sum\_g \sum\_{i \ne j} (x\_{i,g} - \overline{x})(x\_{j,g} - \overline{x}) = \hat \rho\_x \sum\_g \sum\_i (x\_{i,g} - \overline{x})^2 \frac{1}{N} \sum\_g N\_g(N\_g -1)
$$

Which is the sample estimator of $corr(x\_{i,g}, x\_{j,g})$. The numerator of the expression $(1)$ can then be estimated by:
$$
\begin{align\*}
&\sum\_g \sum\_{g'} \sum\_{i \in g} \sum\_{j \in g'} \hat \varepsilon\_{i,g} \hat \varepsilon\_{j,g'} (x\_{i,g}-\overline{x}) (x\_{j,g'}-\overline{x})\\
&\approx \hat \sigma^2 \sum\_g \sum\_{i \in g} (x\_{i,g}- \overline{x})^2 + \hat \sigma^2 \hat \rho \sum\_{g} \sum\_{i \in g} \sum\_{j \in g, i \ne j} (x\_{i,g}-\overline{x})(x\_{j,g} - \overline{x}),\\
&= \hat \sigma^2 \sum\_g \sum\_{i \in g} (x\_{i,g} - \overline{x})^2 + \hat \sigma^2 \hat \rho \hat \rho\_x \frac{1}{N} \sum\_{i}(x\_{i,g} - \overline{x})^2 \sum\_g N\_g(N\_g-1),\\
&=\hat \sigma^2 \sum\_g \sum\_i (x\_{i,g} - \overline{x})^2\left[1 + \left(\frac{1}{N} \sum\_g N\_g^2 - 1\right)\hat \rho \hat \rho\_x \right]
\end{align\*}
$$
Next, observe that:
$$
\begin{align\*}
var(N\_g) &= \frac{1}{G}\sum\_g N\_g^2 - \frac{N^2}{G^2},\\
&\to \frac{1}{N}\sum\_g N\_g^2 = \frac{var(N\_g)}{\overline{m}} + \overline{m}
\end{align\*}
$$
where $\overline{m}$ is the average group size. Then we have that:
$$
\mathbb{E}(\hat \beta - \beta)^2 \approx \frac{\hat \sigma^2 \sum\_g \sum\_{i \in g} (x\_{i,g} - \overline{x})^2 \left(1 + \hat \rho\hat \rho\_x \left(\frac{var(N\_g)}{\overline{m}} + \overline{m} -1\right) \right)}{\left(\sum\_g \sum\_i (x\_i - \overline{x})^2\right)^2} \tag{2}
$$
If we would disregard the clustering, i.e. set $\hat \rho = 0$, we would obtain the estimator:
$$
\mathbb{E}(\hat \beta - \beta)^2 \approx \frac{\hat \sigma^2 \sum\_g \sum\_i (x\_{i,g} - \overline{x})^2}{\left(\sum\_g \sum\_i (x\_{i,g} - \overline{x})^2\right)^2} \tag{3}
$$
The ratio of these two estimators $(2)$ and $(3)$ is then:
$$
\tau = 1 + \left[\frac{var(N\_g)}{\overline{m}} + \overline{m} - 1\right]\hat \rho \hat \rho\_x
$$
This is called the Moulton factor. So if $V(\hat \beta)$ is the estimated variance for the clustered model and $\hat V(\hat \beta)$ is the one for the model that disregards clustering, we have:
$$
V(\hat \beta) = \tau\,\, \hat V(\hat \beta).
$$
We see that $\tau > 1$ if $\hat \rho, \hat \rho\_x > 0$ which will be the case if there is positive intra-group correlation. If the intra-group correlation is negative, but $\rho\_x > 0$, we have that $\tau < 1$. so clustering will decrease the variance as menioned by @username_2

Now, consider the special case where $x\_{i,g}$ only varies over groups. So $x\_{i,g} = x\_g$ for all $i \in g$ and where all groups have the same size. Then we have that $\hat \rho\_x = 1$, $var(N\_g) = 0$ and $\overline{m} = N\_g$. Then:
$$
\tau = 1 + (N\_g - 1)\hat \rho.
$$
So the increase in the estimated variance will be increasing in the intra-group correlation and the number of observations in each group. In general we can expect that $\hat \rho > 0$ so clustered variances will be higher than the standard estimate that does not take into account clustering.


---

