<p>That <a href="https://mathworld.wolfram.com/NormalEquation.html" rel="nofollow noreferrer">normal equation</a> is sometimes called the <a href="https://mathworld.wolfram.com/Closed-FormSolution.html" rel="nofollow noreferrer">closed-form solution</a>.</p>
<p>The short answer to your question is that the closed-form solution may be impractical or unavailable in certain cases or the iterative numerical method (such as gradient descent) may be more efficient (in terms of resources).</p>
<p><a href="https://stats.stackexchange.com/a/23132/82135">This answer</a> gives you more details and an example.</p>

CC BY-SA 4.0
2020-07-08T14:31:12.243Z
2020-07-08T14:31:12.243Z
nbro
<p>Recently, I followed the open course CS229,
<a href="http://cs229.stanford.edu/notes/cs229-notes1.pdf" rel="nofollow noreferrer">http://cs229.stanford.edu/notes/cs229-notes1.pdf</a><br />
This lecturer introduces an alternative approach to gradient descent that is called &quot;Normal Equation&quot; and the equation is as follows:</p>
<p><span class="math-container">$$\theta=\left(X^{T} X\right)^{-1} X^{T} \vec{y}$$</span></p>
<p>The normal equation can directly compute the <span class="math-container">$\theta$</span>.</p>
<p>If the normal equation works, why do we need gradient descent? What is the trade-off between these two methods?</p>

<machine-learning><optimization><gradient-descent>
If the normal equation works, why do we need gradient descent?
Cheleeger Ken
# If the normal equation works, why do we need gradient descent?

(Asked by: username_0 on 2020-07-08)

Recently, I followed the open course CS229,
<http://cs229.stanford.edu/notes/cs229-notes1.pdf>  
This lecturer introduces an alternative approach to gradient descent that is called "Normal Equation" and the equation is as follows:

$$\theta=\left(X^{T} X\right)^{-1} X^{T} \vec{y}$$

The normal equation can directly compute the $\theta$.

If the normal equation works, why do we need gradient descent? What is the trade-off between these two methods?

**Tags:** machine-learning, optimization, gradient-descent


---

## 1 answers

### Answer by username_1 on 2020-07-08. Score: 1 | Accepted answer

That [normal equation](https://mathworld.wolfram.com/NormalEquation.html) is sometimes called the [closed-form solution](https://mathworld.wolfram.com/Closed-FormSolution.html).

The short answer to your question is that the closed-form solution may be impractical or unavailable in certain cases or the iterative numerical method (such as gradient descent) may be more efficient (in terms of resources).

[This answer](https://stats.stackexchange.com/a/23132/82135) gives you more details and an example.


---

