<p>Let's suppose I have $\sqrt{N}$-consistent estimate, $\beta^{*}$(vector, but couldn't put it in bold), then doing the Gauss-Newton Regression, i.e. $\mathbf{y}-\mathbf{x}(\beta^{*})=D_\mathbf{x}(\beta^{*})\mathbf{b}+\mathbf{v}$.</p>

<p>Then we obtain the estimate $\mathbf{\hat b}=\left(D_\mathbf{x}(\beta^{*})^T D_\mathbf{x}(\beta^{*})\right)^{-1}D_\mathbf{x}(\beta^{*})^T(\mathbf{y}-\mathbf{x}(\beta^{*}))$.</p>

<p>By adding this last estimate to $\beta^{*}$, we get the one-step estimator $\bar\beta=\beta^{*}+\mathbf{\hat b}$.</p>

<p>My first question is how the initial estimate is obtained? Do we use the usual method for estimations in this setting (Quasi-Newton methods, etc.)?</p>

<p>Second, how do we prove that this one-step estimator is asymptotically efficient?</p>

<p>I'm following Davidson and Mackinnon (2008) book, but the way they 'derive' it is not really a proof.</p>

<p>So, by trying to mimic their line of reasoning I've done the following:</p>

<p>From a Taylor Expansion/Mean-Value theorem,
$$n^{-1/2}D_\mathbf{x}(\beta^{*})^T(\mathbf{y}-\mathbf{x}(\beta^{*}))=n^{-1/2}D_\mathbf{x}(\beta_0)^T(\mathbf{y}-\mathbf{x}(\beta_0))+\underbrace{\left(o_p(1)-n^{-1}D_\mathbf{x}(\beta)^TD_\mathbf{x}(\beta)\right)}_{\Delta(\beta)}n^{-1/2}(\beta^{*}-\beta_0)$$, where $\beta_0$ is the DGP value of beta, and we've done an abuse of notation by not specifying a different beta for each component of $\Delta(\beta)$. See the answers to <a href="https://economics.stackexchange.com/questions/11516/proving-the-asymptotic-normality-of-th-non-linear-least-squares-estimator">this question</a>, if you're interested in more details on this expansion.</p>

<p>Also, we know from the previous link, that for the Non-linear Least Squares estimator $\hat\beta$, we have $n^{-1/2}(\hat\beta-\beta_0)=-\Delta(\beta)^{-1}n^{-1/2}D_\mathbf{x}(\beta_0)^T\mathbf{u}$.</p>

<p>Hence we can conclude </p>

<p>$n^{-1/2}(\beta^{*}-\beta_0)=\Delta(\beta)^{-1}n^{-1/2}D_\mathbf{x}(\beta^{*})^T(\mathbf{y}-\mathbf{x}(\beta^{*}))+n^{-1/2}(\hat\beta-\beta_0)$, implying that</p>

<p>$n^{-1/2}(\beta^{*}-\hat\beta)=\Delta(\beta)^{-1}n^{-1/2}D_\mathbf{x}(\beta^{*})^T(\mathbf{y}-\mathbf{x}(\beta^{*}))$.</p>

<p>By definition, and some usual assumptions, we know that $plim \Delta(\beta)=S_{D_\mathbf{x}^TD_\mathbf{x}}$. </p>

<p>And I'm stuck here, because what the book does is not translatable when I try be formal in the derivation. So, what should I do to prove that the one-step estimator is efficient?</p>

CC BY-SA 3.0
2016-04-19T13:24:38.200Z
NLLS is an M-estimator, so we can get asymptotic variance by simply using the formula for asymptotic variance of an M-estimator:$$\frac{1}{n}A^{-1}BA^{-1}$$ where $A$ is the expected value of the hessian matrix of the objective function and $B$ is the expected value of the score (jacobian) matrix squared. Because we have a least squares estimator, the objective function is trying to minimize squared errors, so that should give you the objective function.
DornerA
CC BY-SA 3.0
2016-04-19T13:41:34.263Z
@DornerA I was expecting an answer without resorting to that perspective of M-estimators...
An old man in the sea.
CC BY-SA 3.0
2016-04-19T16:18:57.450Z
It was just a suggestion because it is an easy way to find the asymptotic distribution
DornerA
CC BY-SA 3.0
2016-04-19T18:38:09.603Z
@DornerA I think I may have seemed like a bit rude. Sorry. 

Thanks for your comment. I know that we could use the usual derivations for M-estimators, but I'm interested in this particular case, out of curiosity.

Anyhow, thanks for your comment.
An old man in the sea.
<econometrics>
One-Step estimators for non-linear regression
An old man in the sea.
# One-Step estimators for non-linear regression

(Asked by: username_0 on 2016-04-17)

Let's suppose I have $\sqrt{N}$-consistent estimate, $\beta^{\*}$(vector, but couldn't put it in bold), then doing the Gauss-Newton Regression, i.e. $\mathbf{y}-\mathbf{x}(\beta^{\*})=D\_\mathbf{x}(\beta^{\*})\mathbf{b}+\mathbf{v}$.

Then we obtain the estimate $\mathbf{\hat b}=\left(D\_\mathbf{x}(\beta^{\*})^T D\_\mathbf{x}(\beta^{\*})\right)^{-1}D\_\mathbf{x}(\beta^{\*})^T(\mathbf{y}-\mathbf{x}(\beta^{\*}))$.

By adding this last estimate to $\beta^{\*}$, we get the one-step estimator $\bar\beta=\beta^{\*}+\mathbf{\hat b}$.

My first question is how the initial estimate is obtained? Do we use the usual method for estimations in this setting (Quasi-Newton methods, etc.)?

Second, how do we prove that this one-step estimator is asymptotically efficient?

I'm following Davidson and Mackinnon (2008) book, but the way they 'derive' it is not really a proof.

So, by trying to mimic their line of reasoning I've done the following:

From a Taylor Expansion/Mean-Value theorem,
$$n^{-1/2}D\_\mathbf{x}(\beta^{\*})^T(\mathbf{y}-\mathbf{x}(\beta^{\*}))=n^{-1/2}D\_\mathbf{x}(\beta\_0)^T(\mathbf{y}-\mathbf{x}(\beta\_0))+\underbrace{\left(o\_p(1)-n^{-1}D\_\mathbf{x}(\beta)^TD\_\mathbf{x}(\beta)\right)}\_{\Delta(\beta)}n^{-1/2}(\beta^{\*}-\beta\_0)$$, where $\beta\_0$ is the DGP value of beta, and we've done an abuse of notation by not specifying a different beta for each component of $\Delta(\beta)$. See the answers to [this question](https://economics.stackexchange.com/questions/11516/proving-the-asymptotic-normality-of-th-non-linear-least-squares-estimator), if you're interested in more details on this expansion.

Also, we know from the previous link, that for the Non-linear Least Squares estimator $\hat\beta$, we have $n^{-1/2}(\hat\beta-\beta\_0)=-\Delta(\beta)^{-1}n^{-1/2}D\_\mathbf{x}(\beta\_0)^T\mathbf{u}$.

Hence we can conclude

$n^{-1/2}(\beta^{\*}-\beta\_0)=\Delta(\beta)^{-1}n^{-1/2}D\_\mathbf{x}(\beta^{\*})^T(\mathbf{y}-\mathbf{x}(\beta^{\*}))+n^{-1/2}(\hat\beta-\beta\_0)$, implying that

$n^{-1/2}(\beta^{\*}-\hat\beta)=\Delta(\beta)^{-1}n^{-1/2}D\_\mathbf{x}(\beta^{\*})^T(\mathbf{y}-\mathbf{x}(\beta^{\*}))$.

By definition, and some usual assumptions, we know that $plim \Delta(\beta)=S\_{D\_\mathbf{x}^TD\_\mathbf{x}}$.

And I'm stuck here, because what the book does is not translatable when I try be formal in the derivation. So, what should I do to prove that the one-step estimator is efficient?

**Tags:** econometrics

> **Comments:**
> **username_1**: NLLS is an M-estimator, so we can get asymptotic variance by simply using the formula for asymptotic variance of an M-estimator:$$\frac{1}{n}A^{-1}BA^{-1}$$ where $A$ is the expected value of the hessian matrix of the objective function and $B$ is the expected value of the score (jacobian) matrix squared. Because we have a least squares estimator, the objective function is trying to minimize squared errors, so that should give you the objective function.  
> **username_0**: @username_1 I was expecting an answer without resorting to that perspective of M-estimators...  
> **username_1**: It was just a suggestion because it is an easy way to find the asymptotic distribution  
> **username_0**: @username_1 I think I may have seemed like a bit rude. Sorry.   Thanks for your comment. I know that we could use the usual derivations for M-estimators, but I'm interested in this particular case, out of curiosity.  Anyhow, thanks for your comment.  

---

## 0 answers

