<p><strong>What is Error Structure</strong></p>
<p>A good explanation on what error structure is, is already provided in <a href="https://stats.stackexchange.com/a/76481/89502"> this Cross-Validated answer</a>. In short:</p>
<blockquote>
<p>error structure in this respect is referring to the &quot;element of randomness&quot; in your model. For example, in least squares regression, we often assume that the error term of the model (i.e. residuals) follows a normal distribution</p>
</blockquote>
<p>Error structure is basically the catch-all term on all behavior of the error term.</p>
<p><strong>Why Subsampling is Problematic</strong></p>
<p>Now to answer the rest of the question, lets use some arbitrary example where you have sample of both developed and developing countries.</p>
<p>If you assume that both developed and developing countries have the same error structure it does not make much sense to subsample. For example, to make everything simple lets assume  sample where we have 10 developed and 20 developing countries with 15 years worth of observations each (meaning we have 450 observations) and if your model looks like this:</p>
<p><span class="math-container">$$y_{it} =  \alpha_i + \beta_1 T_{it} + \beta_2 X_{it} + \epsilon_{it} \tag{1}$$</span></p>
<p>where <span class="math-container">$y$</span> is outcome, <span class="math-container">$T$</span> some treatment, <span class="math-container">$X$</span> control, and <span class="math-container">$\epsilon$</span> error term.</p>
<p>Now, if we assume error term has the same overall structure it makes sense to examine the extra effect on developed countries in the full sample with interaction variable between treatment and developed country dummy (<span class="math-container">$T \cdot D$</span>).</p>
<p><span class="math-container">$$y_{it} =  \alpha_i + \beta_1 T_{it} + \beta_2 X_{it} + \beta_3 T_{it} \cdot D_i + \epsilon_{it} \tag{2}$$</span></p>
<p>This will tell us exactly how the effect between developed and developing countries differ, since <span class="math-container">$\beta_3$</span> is the extra different effect for developed country, and <span class="math-container">$\beta_1 + \beta_3$</span> gives us the total effect in developed countries, and we got all this by simply sacrificing 1 degree of freedom for this one extra coefficient while our sample size is still 450.  It is important to understand that power of many  statistical tests (such as the <span class="math-container">$t$</span>-test applied to <span class="math-container">$\beta$</span> coefficients) increases in the sample size and decreases when we have more parameters - see discussion of this in Stock and Watson Introduction to Econometrics Ch 3 ). This is because generally speaking the more degrees of freedom you have, the better the power  of statistical test will be. For classical a <span class="math-container">$t$</span>-test degrees of freedom are given by <span class="math-container">$n-k$</span> where <span class="math-container">$n$</span> is the number of observations, <span class="math-container">$k$</span> number of regressors. So in the above example (1) we have <span class="math-container">$n-k= 450- 32 = 418$</span> df (32 because we have 30 dummies for each country and 2 because of 2 other regressors), in the example (2) we would loose only 1 extra df due to interaction term so we would have: <span class="math-container">$n-k=450-33=417$</span>df.</p>
<p>If we would subset the sample we would not get any extra information, and loose more dfs. In this situation we would have separate model for developed countries:</p>
<p><span class="math-container">$$y_{it} =  \omega_i + \gamma_1 T_{it} + \gamma_2 X_{it} + e_{it} \tag{3}$$</span></p>
<p>with 150 observations and 12 parameters (we have only 10 country dummies, so we have <span class="math-container">$150-12= 138$</span>df in model 3, and for developing countries we have</p>
<p><span class="math-container">$$y_{it} =  \kappa_i + \lambda_1 T_{it} + \lambda_2 X_{it} + u_{it} \tag{4}$$</span></p>
<p>with 200 observations and 22 parameters we have <span class="math-container">$200-22=178$</span>df. So by sub-setting the sample we are not gaining any new information since asymptotically <span class="math-container">$\gamma_1 = \beta_1 + \beta_3$</span> and <span class="math-container">$\lambda_1 = \beta_1$</span> from the model (2), but our point estimates of <span class="math-container">$\gamma_1$</span> and <span class="math-container">$\lambda_1$</span> are estimated with less precession, there is higher chance we will not be able to reject null even if alternative hypothesis is true. In addition, to compare the difference between developed and developing countries <span class="math-container">$\gamma_1-\lambda_1$</span> we would have to set up additional test, and in the model (2) we already get this directly from the coefficient <span class="math-container">$\beta_3$</span> since asymptotically <span class="math-container">$\beta_3 = \gamma_1-\lambda_1$</span> (so actually subsampling is even extra work, provided you know how to set up interaction dummies).</p>
<p>However, this is not to say you should never subsample. For example, if the error structure is different, for example, if the developed countries sample suffers from some endogeneity, or if it requires more controls than the developing sample because of omitted variable bias (which affects structure of error term), and if there is no good way to adjust for this problem in whole sample, creating subsamples might be better. If you can justify subsampling then you can do that, but if you just want to see how some variable affects different categories differently, then you should not automatically default to subsampling unless you can justify it by some further arguments.</p>

CC BY-SA 4.0
2021-07-23T12:02:10.503Z
Hi @1muflon1, I am curious why we lose "**simply sacrificing 1 degree of freedom**" here although I understand what is degree of freedom
Phil Nguyen
CC BY-SA 4.0
2021-07-23T12:04:21.440Z
@BeautifulMindset because we are adding 1 extra beta coefficient which means that our system has now one less degree of freedom during the estimation procedure.
1muflon1
CC BY-SA 4.0
2021-07-23T12:04:51.913Z
(2) "**power of many statistical tests decreases when we have more parameters**". Do you mean parameters here are betas ? So, the more beta, the less degree of freedom (df) is what you mean here?
Phil Nguyen
CC BY-SA 4.0
2021-07-23T12:21:57.040Z
(3) And can I ask what does "**precession**" here "**our point estimates of  $\gamma_1$
 and $\lambda_1$ are estimated with less precession**"mean then?
Phil Nguyen
CC BY-SA 4.0
2021-07-23T12:29:46.373Z
@BeautifulMindset 1. yes of course betas are the model parameters by definition of the word parameter. Yes more beta parameters/coefficients (whichever terminology you prefer) the less degrees of freedom the parametric models (such as various regression models) have 2. Precision in statistics refers to the size of standard errors (from which you calculate confidence intervals) if with 95\% confidence $\beta_2 = 2 \pm 10 $ then thats very imprecise estimate as even though point estimate is 2 with 95\% confidence it could be anything between -8 and 12, you typically want as big precision as
1muflon1
CC BY-SA 4.0
2021-07-23T12:31:26.957Z
possible. For example, if with 95\% confidence we can say that $\beta_2=2±0.001$ then thats much more precise, since we know that with 95\% confindece the coefficient is somewhere between 1.999 and 2.001, which is obviously more precise than saying it can be anything from -8 to 12
1muflon1
CC BY-SA 4.0
2021-07-23T12:38:07.803Z
a very comprehensive answer, no more question to ask here, many thanks for your lessons, 1muflon1.
Phil Nguyen
CC BY-SA 4.0
2021-07-22T11:17:09.463Z
2021-07-22T11:17:09.463Z
1muflon1
<p>Subsampling is normally examined in many international studies (grouped by market developement, country governance,etc), @1muflon1 has a great explanation <a href="https://economics.stackexchange.com/a/45902/34781">here</a> in one case.</p>
<p>In his answer, there is one thing I am intersted in that there seems not  enough space in the answer part so I raise a separate answer.</p>
<p>In his answer, he said</p>
<blockquote>
<p>&quot;Unless you have some specific reason to subsample (e.g. you have long
panel an you believe the <strong>error structure</strong> is different between
developed countries so you need special specification for either
subsample) you should not do subsampling.&quot;</p>
</blockquote>
<p>I am wondering what does &quot;error structure&quot; here mean. Doe this statement mean that, we should not subsample the whole sample to developed and developing separately and run the main equation for these both subsamples and then comparing the coefficients between them <strong>because the different in number of observations will swallow the significant level and deter us from making conclusion</strong>?</p>

<econometrics>
Error structure in subsampling
Phil Nguyen
# Error structure in subsampling

(Asked by: username_0 on 2021-07-22)

Subsampling is normally examined in many international studies (grouped by market developement, country governance,etc), @username_1 has a great explanation [here](https://economics.stackexchange.com/a/45902/34781) in one case.

In his answer, there is one thing I am intersted in that there seems not enough space in the answer part so I raise a separate answer.

In his answer, he said

> "Unless you have some specific reason to subsample (e.g. you have long
> panel an you believe the **error structure** is different between
> developed countries so you need special specification for either
> subsample) you should not do subsampling."

I am wondering what does "error structure" here mean. Doe this statement mean that, we should not subsample the whole sample to developed and developing separately and run the main equation for these both subsamples and then comparing the coefficients between them **because the different in number of observations will swallow the significant level and deter us from making conclusion**?

**Tags:** econometrics


---

## 1 answers

### Answer by username_1 on 2021-07-22. Score: 2 | Accepted answer

**What is Error Structure**

A good explanation on what error structure is, is already provided in  [this Cross-Validated answer](https://stats.stackexchange.com/a/76481/89502). In short:

> error structure in this respect is referring to the "element of randomness" in your model. For example, in least squares regression, we often assume that the error term of the model (i.e. residuals) follows a normal distribution

Error structure is basically the catch-all term on all behavior of the error term.

**Why Subsampling is Problematic**

Now to answer the rest of the question, lets use some arbitrary example where you have sample of both developed and developing countries.

If you assume that both developed and developing countries have the same error structure it does not make much sense to subsample. For example, to make everything simple lets assume sample where we have 10 developed and 20 developing countries with 15 years worth of observations each (meaning we have 450 observations) and if your model looks like this:

$$y\_{it} = \alpha\_i + \beta\_1 T\_{it} + \beta\_2 X\_{it} + \epsilon\_{it} \tag{1}$$

where $y$ is outcome, $T$ some treatment, $X$ control, and $\epsilon$ error term.

Now, if we assume error term has the same overall structure it makes sense to examine the extra effect on developed countries in the full sample with interaction variable between treatment and developed country dummy ($T \cdot D$).

$$y\_{it} = \alpha\_i + \beta\_1 T\_{it} + \beta\_2 X\_{it} + \beta\_3 T\_{it} \cdot D\_i + \epsilon\_{it} \tag{2}$$

This will tell us exactly how the effect between developed and developing countries differ, since $\beta\_3$ is the extra different effect for developed country, and $\beta\_1 + \beta\_3$ gives us the total effect in developed countries, and we got all this by simply sacrificing 1 degree of freedom for this one extra coefficient while our sample size is still 450. It is important to understand that power of many statistical tests (such as the $t$-test applied to $\beta$ coefficients) increases in the sample size and decreases when we have more parameters - see discussion of this in Stock and Watson Introduction to Econometrics Ch 3 ). This is because generally speaking the more degrees of freedom you have, the better the power of statistical test will be. For classical a $t$-test degrees of freedom are given by $n-k$ where $n$ is the number of observations, $k$ number of regressors. So in the above example (1) we have $n-k= 450- 32 = 418$ df (32 because we have 30 dummies for each country and 2 because of 2 other regressors), in the example (2) we would loose only 1 extra df due to interaction term so we would have: $n-k=450-33=417$df.

If we would subset the sample we would not get any extra information, and loose more dfs. In this situation we would have separate model for developed countries:

$$y\_{it} = \omega\_i + \gamma\_1 T\_{it} + \gamma\_2 X\_{it} + e\_{it} \tag{3}$$

with 150 observations and 12 parameters (we have only 10 country dummies, so we have $150-12= 138$df in model 3, and for developing countries we have

$$y\_{it} = \kappa\_i + \lambda\_1 T\_{it} + \lambda\_2 X\_{it} + u\_{it} \tag{4}$$

with 200 observations and 22 parameters we have $200-22=178$df. So by sub-setting the sample we are not gaining any new information since asymptotically $\gamma\_1 = \beta\_1 + \beta\_3$ and $\lambda\_1 = \beta\_1$ from the model (2), but our point estimates of $\gamma\_1$ and $\lambda\_1$ are estimated with less precession, there is higher chance we will not be able to reject null even if alternative hypothesis is true. In addition, to compare the difference between developed and developing countries $\gamma\_1-\lambda\_1$ we would have to set up additional test, and in the model (2) we already get this directly from the coefficient $\beta\_3$ since asymptotically $\beta\_3 = \gamma\_1-\lambda\_1$ (so actually subsampling is even extra work, provided you know how to set up interaction dummies).

However, this is not to say you should never subsample. For example, if the error structure is different, for example, if the developed countries sample suffers from some endogeneity, or if it requires more controls than the developing sample because of omitted variable bias (which affects structure of error term), and if there is no good way to adjust for this problem in whole sample, creating subsamples might be better. If you can justify subsampling then you can do that, but if you just want to see how some variable affects different categories differently, then you should not automatically default to subsampling unless you can justify it by some further arguments.

> **Comments:**
> **username_0**: Hi @username_1, I am curious why we lose "**simply sacrificing 1 degree of freedom**" here although I understand what is degree of freedom  
> **username_1**: @BeautifulMindset because we are adding 1 extra beta coefficient which means that our system has now one less degree of freedom during the estimation procedure.  
> **username_0**: (2) "**power of many statistical tests decreases when we have more parameters**". Do you mean parameters here are betas ? So, the more beta, the less degree of freedom (df) is what you mean here?  
> **username_0**: (3) And can I ask what does "**precession**" here "**our point estimates of  $\gamma_1$  and $\lambda_1$ are estimated with less precession**"mean then?  
> **username_1**: @BeautifulMindset 1. yes of course betas are the model parameters by definition of the word parameter. Yes more beta parameters/coefficients (whichever terminology you prefer) the less degrees of freedom the parametric models (such as various regression models) have 2. Precision in statistics refers to the size of standard errors (from which you calculate confidence intervals) if with 95\% confidence $\beta_2 = 2 \pm 10 $ then thats very imprecise estimate as even though point estimate is 2 with 95\% confidence it could be anything between -8 and 12, you typically want as big precision as  
> **username_1**: possible. For example, if with 95\% confidence we can say that $\beta_2=2±0.001$ then thats much more precise, since we know that with 95\% confindece the coefficient is somewhere between 1.999 and 2.001, which is obviously more precise than saying it can be anything from -8 to 12  
> **username_0**: a very comprehensive answer, no more question to ask here, many thanks for your lessons, username_1.  

---

