<p>I will provide another hint, more "primitive" than the one offered by @Anoldmaninthesea, for those that are not still very familiar with the "big-Oh/little oh" notation and arithmetic.  </p>

<p>What we are examining here is a sum, which, following the first obvious "hint" given, is to be decomposed in <em>three</em> separate sums.  </p>

<p>Now, we are interested in what happens to the value of these sums as the number of summands goes to infinity... In such cases, essentially we are looking if "we have enough "$N$"'s for the infinite sum to go in value to something finite (given the a priori assumptions), and if yes, whether this finite limit is zero or not.  </p>

<p>Consider the middle sum</p>

<p>$$\frac{1}{n} \sum\limits_{i=1}^{n}[-2x_iu_i(\hat{\beta_n}-\beta)\mathbf x_i'\mathbf x_i] = -2(\hat{\beta_n}-\beta)\frac{1}{n} \sum\limits_{i=1}^{n}(x_iu_i\mathbf x_i'\mathbf x_i)$$</p>

<p>$(\hat{\beta_n}-\beta)$ was taken out of the sum because it does not depend on the <em>index</em> $i$. It does depend on $n$, since it is an estimator function and not a specific estimate, but not on $i$.
The assumptions of the model tell us what happens to $(\hat{\beta_n}-\beta)$, if it remains unscaled, as $n\rightarrow \infty$. As for the summands, $(\mathbf x_i'\mathbf x_i)$ is a $k\times k$ matrix, that does not include itself any sum. So it is finite (the fact that the random variables involved have possibly infinite support does not bother us -the probability that they will take the value "infinity" is zero). So it is only the whole expression $(x_iu_i\mathbf x_i'\mathbf x_i)$ that multiplies in cardinality as $n$ increases - and by the assumptions of the model we know things about $x_i$ and $u_i$ and what happens to the average of their product. Etc.  </p>

<p>An unrelated example to show what this "how many $n$'s" business means is to consider, for example, a random vector like</p>

<p>$$\frac{1}{\sqrt n}\left(\mathbf X'\mathbf X (\hat \beta_n - \beta)\right)$$</p>

<p>We "know" that this random vector converges to a multivariate distribution, i.e. it neither converges to a constant, nor does it diverge. Can we see this in our primitive way? We would want a finite asymptotic variance. The asymptotic mean will be zero so we need only to examine </p>

<p>$$V_n=E\left[\frac{1}{\sqrt n}\mathbf X'\mathbf X(\hat \beta_n - \beta)(\hat \beta_n - \beta)'\mathbf X'\mathbf X\frac{1}{\sqrt n}\right] = E\left[\left(\frac{1}{n}\mathbf X'\mathbf X\right)[(\hat \beta_n - \beta)(\hat \beta_n - \beta)']\left(\mathbf X' \mathbf X\right)\right]$$</p>

<p>What is the problem with the above expectation? The first term wll converge to something finite and non-zero (per initial assumptions), the second goes to zero, the third goes to infinity(because it includes sums whose cardinality will go to infinity). So the expectation is still of undetermined fate. But if we multiply and divide by $n$ (we <em>can</em> do that, we have not yet send $n$ to infinity) we obtain</p>

<p>$$V_n= E\left[\left(\frac{1}{n}\mathbf X'\mathbf X\right)[n\cdot(\hat \beta_n - \beta)(\hat \beta_n - \beta)']\left(\frac{1}{n}\mathbf X' \mathbf X\right)\right]$$</p>

<p>Now things begin to clear: the first and second terms will converge to expected values, <em>so they can be taken out of the external expected value</em>:</p>

<p>$$V_n\rightarrow \left(\lim_{n\rightarrow \infty} E\frac{1}{n}\mathbf X'\mathbf X\right)\lim_{n\rightarrow \infty} E\left[n\cdot(\hat \beta_n - \beta)(\hat \beta_n - \beta)'\right]\left(\lim_{n\rightarrow \infty}E\frac{1}{n}\mathbf X' \mathbf X\right)$$</p>

<p>The middle term inside the expectation  is the (multivariate) "square" of $\sqrt n (\hat \beta_n - \beta)$. We know that this last one converges to a zero-mean random vector, so the limit of the expected value of its "square" is its asymptotic variance, which is finite. So we have the product of three asymptotically finite expected values, and so the whole expression is finite, and so the variance of the expression we started with is finite, and moreover, non-zero (by the usual initial assumptions of the model).</p>

CC BY-SA 3.0
2015-09-08T13:01:39.607Z
2015-09-08T13:01:39.607Z
Alecos Papadopoulos
<p>In hint 1 substitute $\hat{\beta}-\beta$ by little o, and remember that the sum of little o 's is still little o's. Then substitute on the LHS of your equality abouve. and you get what you want. (multiplication of $x'x$ by little o is also little o) I hope this is enough. If not, I'll show more later. I'm pressed for time right now... sorry</p>

CC BY-SA 3.0
2015-09-08T08:38:51.287Z
2015-09-08T08:38:51.287Z
An old man in the sea.
<p>The following exercise is from Wooldrige: </p>

<p>Show that $\hat{\beta} =\frac{1}{N} \sum\limits_{i=1}^{N}\hat{u_i^{2}}x'x $ is a consistent estimator for $E(u^2x'x)$</p>

<p>by showing: </p>

<p>$$\frac{1}{N} \sum\limits_{i=1}^{N}\hat{u_i^{2}}x'x = \frac{1}{N} \sum\limits_{i=1}^{N}{u_i^{2}}x'x + o_p(1) $$ </p>

<p>Where $o_p(1)$ is read "little oh p 1" </p>

<p>And we use the hints that: </p>

<p>1.) $\hat{u}_i^2$ = $u_i^2 - 2x_iu_i(\hat{\beta}-\beta) + [x_i(\hat{\beta}-\beta)]^2$</p>

<p>2.) $\hat{\beta} - \beta$ = $o_p(1)$ </p>

<p>3.)Sample averages are $O_p(1)$  read as "Big Oh P 1" </p>

<p>4.) we assume all necessary expectations exist and are finite </p>

<p>I'm getting stuck with this. I know I must be missing some simple sort of substitution or there is some gap in my knowledge or understanding preventing me from making the necessary manipulations.  </p>

<p>I would love if someone could walk me through this and explain the intuition a bit here. </p>

CC BY-SA 3.0
2015-09-08T07:40:48.023Z
I'm voting to close this question as off-topic because this is an exercise and there is no effort towards solution shown.
Giskard
CC BY-SA 3.0
2015-09-08T07:48:12.170Z
I've been working toward a solution on my whiteboard for the past two hours. It's a bit ridiculous for you to assume a person addressing a question of this level is the type of person who wouldn't want to solve it himself or herself.
123
CC BY-SA 3.0
2015-09-08T07:53:48.747Z
I think the standard text was "homework question", not "exercise". For some reason (perhaps this is to be discussed in meta), I personally don't require more advanced questions to have the same effort in solving it shown. Perhaps it is because advanced questions are more interesting intrinsically.
FooBar
CC BY-SA 3.0
2015-09-08T07:59:26.573Z
I omitted my efforts largely because I doubt any reader will gain much from them. And my chosen path is relatively obvious: I follow the hints, of course.
123
CC BY-SA 3.0
2015-09-08T08:18:01.357Z
Very well. Perhaps I misjudged the level if this exercise. In this case there will be more votes against than for closing, which is fine. I think this is one of the reasons why the system requires 5 votes to close.
Giskard
CC BY-SA 3.0
2015-09-08T18:04:52.857Z
@mathtastic it is always better to put on the site what you have tried. People here try to get the signal that asked questions are not homework questions, without any effort made.
optimal control
<econometrics>
Prove that an estimator is consistent
123
# Prove that an estimator is consistent

(Asked by: username_0 on 2015-09-08)

The following exercise is from Wooldrige:

Show that $\hat{\beta} =\frac{1}{N} \sum\limits\_{i=1}^{N}\hat{u\_i^{2}}x'x $ is a consistent estimator for $E(u^2x'x)$

by showing:

$$\frac{1}{N} \sum\limits\_{i=1}^{N}\hat{u\_i^{2}}x'x = \frac{1}{N} \sum\limits\_{i=1}^{N}{u\_i^{2}}x'x + o\_p(1) $$

Where $o\_p(1)$ is read "little oh p 1"

And we use the hints that:

1.) $\hat{u}\_i^2$ = $u\_i^2 - 2x\_iu\_i(\hat{\beta}-\beta) + [x\_i(\hat{\beta}-\beta)]^2$

2.) $\hat{\beta} - \beta$ = $o\_p(1)$

3.)Sample averages are $O\_p(1)$ read as "Big Oh P 1"

4.) we assume all necessary expectations exist and are finite

I'm getting stuck with this. I know I must be missing some simple sort of substitution or there is some gap in my knowledge or understanding preventing me from making the necessary manipulations.

I would love if someone could walk me through this and explain the intuition a bit here.

**Tags:** econometrics

> **Comments:**
> **username_1**: I'm voting to close this question as off-topic because this is an exercise and there is no effort towards solution shown.  
> **username_0**: I've been working toward a solution on my whiteboard for the past two hours. It's a bit ridiculous for you to assume a person addressing a question of this level is the type of person who wouldn't want to solve it himself or herself.  
> **username_2**: I think the standard text was "homework question", not "exercise". For some reason (perhaps this is to be discussed in meta), I personally don't require more advanced questions to have the same effort in solving it shown. Perhaps it is because advanced questions are more interesting intrinsically.  
> **username_0**: I omitted my efforts largely because I doubt any reader will gain much from them. And my chosen path is relatively obvious: I follow the hints, of course.  
> **username_1**: Very well. Perhaps I misjudged the level if this exercise. In this case there will be more votes against than for closing, which is fine. I think this is one of the reasons why the system requires 5 votes to close.  
> **username_3**: @mathtastic it is always better to put on the site what you have tried. People here try to get the signal that asked questions are not homework questions, without any effort made.  

---

## 2 answers

### Answer by username_4 on 2015-09-08. Score: 3

I will provide another hint, more "primitive" than the one offered by @Anoldmaninthesea, for those that are not still very familiar with the "big-Oh/little oh" notation and arithmetic.

What we are examining here is a sum, which, following the first obvious "hint" given, is to be decomposed in *three* separate sums.

Now, we are interested in what happens to the value of these sums as the number of summands goes to infinity... In such cases, essentially we are looking if "we have enough "$N$"'s for the infinite sum to go in value to something finite (given the a priori assumptions), and if yes, whether this finite limit is zero or not.

Consider the middle sum

$$\frac{1}{n} \sum\limits\_{i=1}^{n}[-2x\_iu\_i(\hat{\beta\_n}-\beta)\mathbf x\_i'\mathbf x\_i] = -2(\hat{\beta\_n}-\beta)\frac{1}{n} \sum\limits\_{i=1}^{n}(x\_iu\_i\mathbf x\_i'\mathbf x\_i)$$

$(\hat{\beta\_n}-\beta)$ was taken out of the sum because it does not depend on the *index* $i$. It does depend on $n$, since it is an estimator function and not a specific estimate, but not on $i$.
The assumptions of the model tell us what happens to $(\hat{\beta\_n}-\beta)$, if it remains unscaled, as $n\rightarrow \infty$. As for the summands, $(\mathbf x\_i'\mathbf x\_i)$ is a $k\times k$ matrix, that does not include itself any sum. So it is finite (the fact that the random variables involved have possibly infinite support does not bother us -the probability that they will take the value "infinity" is zero). So it is only the whole expression $(x\_iu\_i\mathbf x\_i'\mathbf x\_i)$ that multiplies in cardinality as $n$ increases - and by the assumptions of the model we know things about $x\_i$ and $u\_i$ and what happens to the average of their product. Etc.

An unrelated example to show what this "how many $n$'s" business means is to consider, for example, a random vector like

$$\frac{1}{\sqrt n}\left(\mathbf X'\mathbf X (\hat \beta\_n - \beta)\right)$$

We "know" that this random vector converges to a multivariate distribution, i.e. it neither converges to a constant, nor does it diverge. Can we see this in our primitive way? We would want a finite asymptotic variance. The asymptotic mean will be zero so we need only to examine

$$V\_n=E\left[\frac{1}{\sqrt n}\mathbf X'\mathbf X(\hat \beta\_n - \beta)(\hat \beta\_n - \beta)'\mathbf X'\mathbf X\frac{1}{\sqrt n}\right] = E\left[\left(\frac{1}{n}\mathbf X'\mathbf X\right)[(\hat \beta\_n - \beta)(\hat \beta\_n - \beta)']\left(\mathbf X' \mathbf X\right)\right]$$

What is the problem with the above expectation? The first term wll converge to something finite and non-zero (per initial assumptions), the second goes to zero, the third goes to infinity(because it includes sums whose cardinality will go to infinity). So the expectation is still of undetermined fate. But if we multiply and divide by $n$ (we *can* do that, we have not yet send $n$ to infinity) we obtain

$$V\_n= E\left[\left(\frac{1}{n}\mathbf X'\mathbf X\right)[n\cdot(\hat \beta\_n - \beta)(\hat \beta\_n - \beta)']\left(\frac{1}{n}\mathbf X' \mathbf X\right)\right]$$

Now things begin to clear: the first and second terms will converge to expected values, *so they can be taken out of the external expected value*:

$$V\_n\rightarrow \left(\lim\_{n\rightarrow \infty} E\frac{1}{n}\mathbf X'\mathbf X\right)\lim\_{n\rightarrow \infty} E\left[n\cdot(\hat \beta\_n - \beta)(\hat \beta\_n - \beta)'\right]\left(\lim\_{n\rightarrow \infty}E\frac{1}{n}\mathbf X' \mathbf X\right)$$

The middle term inside the expectation is the (multivariate) "square" of $\sqrt n (\hat \beta\_n - \beta)$. We know that this last one converges to a zero-mean random vector, so the limit of the expected value of its "square" is its asymptotic variance, which is finite. So we have the product of three asymptotically finite expected values, and so the whole expression is finite, and so the variance of the expression we started with is finite, and moreover, non-zero (by the usual initial assumptions of the model).


---

### Answer by username_5 on 2015-09-08. Score: 1

In hint 1 substitute $\hat{\beta}-\beta$ by little o, and remember that the sum of little o 's is still little o's. Then substitute on the LHS of your equality abouve. and you get what you want. (multiplication of $x'x$ by little o is also little o) I hope this is enough. If not, I'll show more later. I'm pressed for time right now... sorry


---

