<p>The conditions you derive guarantee that <span class="math-container">$(\hat{\alpha}, \hat{\beta})$</span> occur where SSE is locally minimized. Since our estimates are unique, i.e. there is a unique parameter vector that satisfies our first-order conditions, we know the selected parameter vector minimizes the objective function in the interior of the parameter space.</p>

<p>To determine whether this is a global minimum, you would compare the SSE under our estimates to the boundary points. For a simple linear regression as you've described, the parameter space is <span class="math-container">$\mathbb{R}^2$</span>, and is therefore unbounded so <span class="math-container">$(\hat{\alpha}, \hat{\beta})$</span> globally minimizes SSE. In contrast, if you had, for example, the restriction that a parameter was non-negative, checking the boundary condition would matter for determining a global minimum of SSE. </p>

CC BY-SA 4.0
2019-03-19T17:01:07.480Z
When I said the conditions you derive, I meant the first-order conditions and the second-order conditions.
dlnB
CC BY-SA 4.0
2019-03-18T18:09:12.120Z
2019-03-18T18:15:33.830Z
2019-03-18T18:15:33.830Z
dlnB
<p>In deriving our ordinary least squares estimates, we can partially differentiate the sum of squared errors <span class="math-container">$\sum_{i=1}^{n} {e_i^2} = \sum_{i=1}^{n} {(Y_i- \hat{\alpha}-\hat{\beta}X_i )^2}$</span> with respect to our estimators <span class="math-container">$\hat{\alpha}$</span> and <span class="math-container">$\hat{\beta}$</span> to get the following first order conditions:</p>

<p><span class="math-container">$$\frac{\delta SSR}{\delta \hat{\alpha}}= -2\sum_{i=1}^{n} {e_i}=-2\sum_{i=1}^{n} {(Y_i- \hat{\alpha}-\hat{\beta}X_i)}=0$$</span>
<span class="math-container">$$\frac{\delta SSR}{\delta \hat{\beta}}= -2\sum_{i=1}^{n} {X_i e_i}=-2\sum_{i=1}^{n} X_i {(Y_i- \hat{\alpha}-\hat{\beta}X_i)}=0$$</span></p>

<p>How can we then prove that the sum of squared residuals is a global minimum?</p>

<p>Firstly, can I confirm that the Hessian looks like this?</p>

<p><span class="math-container">\begin{array}{cc}
   2n &amp; 2\sum_{i=1}^{n}X_i  \\
   2\sum_{i=1}^{n}X_i  &amp; 2\sum_{i=1}^{n}(X_i)^2  \\
  \end{array}</span></p>

<p>Secondly, how can I show that the determinant of this Hessian is positive definite, i.e. <span class="math-container">$ 4[n \sum_{i=1}^{n}(X_i)^2 - (\sum_{i=1}^{n}X_i)^2]&gt;0?$</span> The presentation bears some resemblance to the population variance formula, and I know that variance must be non-negative, but I'm wondering how to move on. Is there some way to factorise this? Thank you.</p>

<regression>
OLS estimator derivation: second-order condition to prove global minimum?
rchng
# OLS estimator derivation: second-order condition to prove global minimum?

(Asked by: username_0 on 2019-03-18)

In deriving our ordinary least squares estimates, we can partially differentiate the sum of squared errors $\sum\_{i=1}^{n} {e\_i^2} = \sum\_{i=1}^{n} {(Y\_i- \hat{\alpha}-\hat{\beta}X\_i )^2}$ with respect to our estimators $\hat{\alpha}$ and $\hat{\beta}$ to get the following first order conditions:

$$\frac{\delta SSR}{\delta \hat{\alpha}}= -2\sum\_{i=1}^{n} {e\_i}=-2\sum\_{i=1}^{n} {(Y\_i- \hat{\alpha}-\hat{\beta}X\_i)}=0$$
$$\frac{\delta SSR}{\delta \hat{\beta}}= -2\sum\_{i=1}^{n} {X\_i e\_i}=-2\sum\_{i=1}^{n} X\_i {(Y\_i- \hat{\alpha}-\hat{\beta}X\_i)}=0$$

How can we then prove that the sum of squared residuals is a global minimum?

Firstly, can I confirm that the Hessian looks like this?

\begin{array}{cc}
2n & 2\sum\_{i=1}^{n}X\_i \\
2\sum\_{i=1}^{n}X\_i & 2\sum\_{i=1}^{n}(X\_i)^2 \\
\end{array}

Secondly, how can I show that the determinant of this Hessian is positive definite, i.e. $ 4[n \sum\_{i=1}^{n}(X\_i)^2 - (\sum\_{i=1}^{n}X\_i)^2]>0?$ The presentation bears some resemblance to the population variance formula, and I know that variance must be non-negative, but I'm wondering how to move on. Is there some way to factorise this? Thank you.

**Tags:** regression


---

## 1 answers

### Answer by username_1 on 2019-03-18. Score: 1

The conditions you derive guarantee that $(\hat{\alpha}, \hat{\beta})$ occur where SSE is locally minimized. Since our estimates are unique, i.e. there is a unique parameter vector that satisfies our first-order conditions, we know the selected parameter vector minimizes the objective function in the interior of the parameter space.

To determine whether this is a global minimum, you would compare the SSE under our estimates to the boundary points. For a simple linear regression as you've described, the parameter space is $\mathbb{R}^2$, and is therefore unbounded so $(\hat{\alpha}, \hat{\beta})$ globally minimizes SSE. In contrast, if you had, for example, the restriction that a parameter was non-negative, checking the boundary condition would matter for determining a global minimum of SSE.

> **Comments:**
> **username_1**: When I said the conditions you derive, I meant the first-order conditions and the second-order conditions.  

---

