<p>If the regressors are <strong>strictly</strong> exogenous to the error term (which is the initial standard assumption made), meaning, if the error <em>vector</em> is mean-independent of the regressor <em>matrix</em></p>

<p>$$E(\mathbf u \mid \mathbf X) = \mathbf 0$$</p>

<p>then consistency follows immediately. Set for convenience $\mathbf \Omega ^{-1/2} \equiv \mathbf C$. Then </p>

<p>$$\text{plim}\left(n^{-1} (\mathbf C\mathbf X)'\mathbf u \right)= \text{plim}\left(n^{-1}\mathbf X'\mathbf C'\mathbf u\right)$$</p>

<p>This is a $k \times 1$ matrix. What would a typical element will be, say the first?  </p>

<p>The matrix $\mathbf C'\mathbf u$ is a $n\times 1$ vector,</p>

<p>$$\mathbf C'\mathbf u = \left(\sum_{i=1}^nc_{1i}u_i,..., \sum_{i=1}^nc_{ni}u_i\right)'$$</p>

<p>Mutliplied by the first row of $\mathbf X' = (x_{11}\;...\;x_{1n})$ gives</p>

<p>$$\left [\mathbf X'\mathbf C'\mathbf u\right]_1 =x_{11}\cdot \sum_{i=1}^nc_{1i}u_i\;+...\;+ x_{1n}\cdot\sum_{i=1}^nc_{ni}u_i$$</p>

<p>Now strict exogeneity comes into play: it is stronger and so implies that </p>

<p>$$E(\mathbf u \mid \mathbf X) = \mathbf 0 \implies E(x_{1i}u_j)=0\;\;\forall i,j$$ </p>

<p>So the above sum of sums, scaled by $n^{-1}$ will tend to zero asymptotically, since all products in each sum will tend to a zero expected value. This proves consistency.</p>

<p>Note that strict exogeneity is really needed, for the case where the matrix $\Omega$ is a function of the regressors. In such a case, we would be looking at products of the form</p>

<p>$$x_{1i}\cdot h(\mathbf x) \cdot u_j$$</p>

<p>which would <em>still</em> go to zero in expected value terms, because mean-independence implies orthogonality between the error term and <em>any</em> function of the regressors.</p>

<p>If $\Omega$ does not depend on the regressors, then we could weaken strict exogeneity to the RHS condition, which is  stronger than "contemporaneous uncorrelatedness", the latter needed for OLS. </p>

<p>I  leave the case for asymptotic normality to anyone interested.</p>

CC BY-SA 4.0
2019-07-13T14:59:02.547Z
Misleading to claim the above is an argument for consistency. "...So the above sum of sums, scaled by $n^{−1}$ will tend to zero asymptotically..." is clearly not true in general, even in the i.i.d. case. It's a sum of $n$ sums, and $n \rightarrow \infty$. Expectation and infinite series already doesn't commute in general, much less probability limit and infinite series. Counterexamples are easy to construct with diagonal $\Omega$. As OP already pointed out in the comments, this requires boundedness-from-below type of assumptions on $\frac{\Omega^{-1}}{n}$ .
Michael
CC BY-SA 4.0
2019-07-14T16:52:06.677Z
@Michael Hmm, I guess I left certain "usual" assumptions implied and not explicitly stated. Let me see how I can upgrade the quality of this answer.
Alecos Papadopoulos
CC BY-SA 3.0
2017-04-16T14:51:58.400Z
2017-04-16T14:51:58.400Z
Alecos Papadopoulos
<p>$$\mathbf Y=\mathbf X\beta + \mathbf U$$
$$\mathbf E[\mathbf U\mathbf U']=\Omega $$</p>

<p>$$\hat \beta - \beta=(\mathbf X'\Omega^{-1}\mathbf X)^{-1}\mathbf X'\Omega^{-1}\mathbf U$$</p>

<p>let $\mathbf X^{*}=\Omega^{-0.5}\mathbf X$, and $\mathbf U^{*}=\Omega^{-0.5}\mathbf U$. it is obviously that $\mathbf u^{*}_i$,which is the i-th element of $\mathbf U^{*}$, is iid follows the standard normal distribution, if we assume that $\mathbf U$ follows normal distribution. But $x_i^{*'}$, the i-th row of $\mathbf X^{*}$ is not necessarily independent, for it is the linear combination of all rows in $\mathbf X$ (which is assumed iid for they stand for observations from the same distribution).</p>

<p>Thus the followings are not necessarily true under independent Law of Large number and Central limit theorem:</p>

<ol>
<li><p>$plim\frac{\mathbf X'\Omega^{-1}\mathbf U}{n}=plim\frac{\mathbf X^{*'}\mathbf U^{*}}{n}= plim\frac{1}{n}\Sigma x_i^{*}u_i^{*} =0$</p></li>
<li><p>$\frac{\mathbf X'\Omega^{-1}\mathbf U}{\sqrt n}=\frac{\mathbf X^{*'}\mathbf U^{*}}{\sqrt n}= \frac{1}{\sqrt n}\Sigma x_i^{*}u_i^{*} $, converges to normal distribution</p></li>
</ol>

<p>I understand that when $\mathbf \Omega $ is a diagonal matrix then $x_i^{*}$ is independent and it is not a problem, what if when $\mathbf \Omega $ is not a diagonal matrix? What kind of dependent LLN and CLT can be applied here? should we make other assumptions?</p>

CC BY-SA 3.0
2016-04-21T06:43:04.990Z
If you have iid sample, the regressors are exogen and $\Omega$ (which is really a sequence indexed by sample size) is uniformly positive definite, and has a block diagonal structure, then consistency follows from LLN for, e.g. mixing processes. The moment restrictions, e.g. finite second moment must exists, etc, for the iid case may need to be strengthened accordingly.
Michael
CC BY-SA 3.0
2016-04-21T10:59:52.440Z
@Michael Assume iid sample and exogeneity. It sounds like there must be some assumptions applied to the $\Omega$ before we could use any LLN and CLT isn't it?
Edward
CC BY-SA 3.0
2016-04-22T03:13:34.227Z
Yes, uniformly positive definite and block diagonal should do it. Although block diagonal may be too strong, it's easier to state, I will try to post an answer if someone doesn't beat me to it.
Michael
CC BY-SA 3.0
2016-04-29T07:29:41.360Z
@Michael I just come up with a solution by the Chebyshev's inequality. P[|$\frac{\mathbf X'\Omega^{-1}\mathbf U}{n}$ |>$\epsilon$]<= $\frac{E[\mathbf X'\Omega^{-1}\mathbf U\mathbf U'\Omega^{-1}\mathbf X']}{n^2}$ =$\frac{E[\mathbf X'\Omega^{-1}\mathbf X']}{n^2}$ . This goes to zero if we only assume finite second moment of X. Thus the consistency is proved. Am I right?
Edward
<econometrics>
How to show that GLS estimator is consistent in regression model?
Edward
# How to show that GLS estimator is consistent in regression model?

(Asked by: username_0 on 2016-04-17)

$$\mathbf Y=\mathbf X\beta + \mathbf U$$
$$\mathbf E[\mathbf U\mathbf U']=\Omega $$

$$\hat \beta - \beta=(\mathbf X'\Omega^{-1}\mathbf X)^{-1}\mathbf X'\Omega^{-1}\mathbf U$$

let $\mathbf X^{\*}=\Omega^{-0.5}\mathbf X$, and $\mathbf U^{\*}=\Omega^{-0.5}\mathbf U$. it is obviously that $\mathbf u^{\*}\_i$,which is the i-th element of $\mathbf U^{\*}$, is iid follows the standard normal distribution, if we assume that $\mathbf U$ follows normal distribution. But $x\_i^{\*'}$, the i-th row of $\mathbf X^{\*}$ is not necessarily independent, for it is the linear combination of all rows in $\mathbf X$ (which is assumed iid for they stand for observations from the same distribution).

Thus the followings are not necessarily true under independent Law of Large number and Central limit theorem:

1. $plim\frac{\mathbf X'\Omega^{-1}\mathbf U}{n}=plim\frac{\mathbf X^{\*'}\mathbf U^{\*}}{n}= plim\frac{1}{n}\Sigma x\_i^{\*}u\_i^{\*} =0$
2. $\frac{\mathbf X'\Omega^{-1}\mathbf U}{\sqrt n}=\frac{\mathbf X^{\*'}\mathbf U^{\*}}{\sqrt n}= \frac{1}{\sqrt n}\Sigma x\_i^{\*}u\_i^{\*} $, converges to normal distribution

I understand that when $\mathbf \Omega $ is a diagonal matrix then $x\_i^{\*}$ is independent and it is not a problem, what if when $\mathbf \Omega $ is not a diagonal matrix? What kind of dependent LLN and CLT can be applied here? should we make other assumptions?

**Tags:** econometrics

> **Comments:**
> **username_1**: If you have iid sample, the regressors are exogen and $\Omega$ (which is really a sequence indexed by sample size) is uniformly positive definite, and has a block diagonal structure, then consistency follows from LLN for, e.g. mixing processes. The moment restrictions, e.g. finite second moment must exists, etc, for the iid case may need to be strengthened accordingly.  
> **username_0**: @username_1 Assume iid sample and exogeneity. It sounds like there must be some assumptions applied to the $\Omega$ before we could use any LLN and CLT isn't it?  
> **username_1**: Yes, uniformly positive definite and block diagonal should do it. Although block diagonal may be too strong, it's easier to state, I will try to post an answer if someone doesn't beat me to it.  
> **username_0**: @username_1 I just come up with a solution by the Chebyshev's inequality. P[|$\frac{\mathbf X'\Omega^{-1}\mathbf U}{n}$ |>$\epsilon$]<= $\frac{E[\mathbf X'\Omega^{-1}\mathbf U\mathbf U'\Omega^{-1}\mathbf X']}{n^2}$ =$\frac{E[\mathbf X'\Omega^{-1}\mathbf X']}{n^2}$ . This goes to zero if we only assume finite second moment of X. Thus the consistency is proved. Am I right?  

---

## 1 answers

### Answer by username_2 on 2017-04-16. Score: 4

If the regressors are **strictly** exogenous to the error term (which is the initial standard assumption made), meaning, if the error *vector* is mean-independent of the regressor *matrix*

$$E(\mathbf u \mid \mathbf X) = \mathbf 0$$

then consistency follows immediately. Set for convenience $\mathbf \Omega ^{-1/2} \equiv \mathbf C$. Then

$$\text{plim}\left(n^{-1} (\mathbf C\mathbf X)'\mathbf u \right)= \text{plim}\left(n^{-1}\mathbf X'\mathbf C'\mathbf u\right)$$

This is a $k \times 1$ matrix. What would a typical element will be, say the first?

The matrix $\mathbf C'\mathbf u$ is a $n\times 1$ vector,

$$\mathbf C'\mathbf u = \left(\sum\_{i=1}^nc\_{1i}u\_i,..., \sum\_{i=1}^nc\_{ni}u\_i\right)'$$

Mutliplied by the first row of $\mathbf X' = (x\_{11}\;...\;x\_{1n})$ gives

$$\left [\mathbf X'\mathbf C'\mathbf u\right]\_1 =x\_{11}\cdot \sum\_{i=1}^nc\_{1i}u\_i\;+...\;+ x\_{1n}\cdot\sum\_{i=1}^nc\_{ni}u\_i$$

Now strict exogeneity comes into play: it is stronger and so implies that

$$E(\mathbf u \mid \mathbf X) = \mathbf 0 \implies E(x\_{1i}u\_j)=0\;\;\forall i,j$$

So the above sum of sums, scaled by $n^{-1}$ will tend to zero asymptotically, since all products in each sum will tend to a zero expected value. This proves consistency.

Note that strict exogeneity is really needed, for the case where the matrix $\Omega$ is a function of the regressors. In such a case, we would be looking at products of the form

$$x\_{1i}\cdot h(\mathbf x) \cdot u\_j$$

which would *still* go to zero in expected value terms, because mean-independence implies orthogonality between the error term and *any* function of the regressors.

If $\Omega$ does not depend on the regressors, then we could weaken strict exogeneity to the RHS condition, which is stronger than "contemporaneous uncorrelatedness", the latter needed for OLS.

I leave the case for asymptotic normality to anyone interested.

> **Comments:**
> **username_1**: Misleading to claim the above is an argument for consistency. "...So the above sum of sums, scaled by $n^{−1}$ will tend to zero asymptotically..." is clearly not true in general, even in the i.i.d. case. It's a sum of $n$ sums, and $n \rightarrow \infty$. Expectation and infinite series already doesn't commute in general, much less probability limit and infinite series. Counterexamples are easy to construct with diagonal $\Omega$. As OP already pointed out in the comments, this requires boundedness-from-below type of assumptions on $\frac{\Omega^{-1}}{n}$ .  
> **username_2**: @username_1 Hmm, I guess I left certain "usual" assumptions implied and not explicitly stated. Let me see how I can upgrade the quality of this answer.  

---

