Answers.Body: <p>That <a href="https://mathworld.wolfram.com/NormalEquation.html" rel="nofollow noreferrer">normal equation</a> is sometimes called the <a href="https://mathworld.wolfram.com/Closed-FormSolution.html" rel="nofollow noreferrer">closed-form solution</a>.</p>
  <p>The short answer to your question is that the closed-form solution may be impractical or unavailable in certain cases or the iterative numerical method (such as gradient descent) may be more efficient (in terms of resources).</p>
  <p><a href="https://stats.stackexchange.com/a/23132/82135">This answer</a> gives you more details and an example.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-07-08T14:31:12.243Z
Answers.LastActivityDate: 2020-07-08T14:31:12.243Z
Answers.UserDisplayName: nbro
Body: <p>Recently, I followed the open course CS229,
  <a href="http://cs229.stanford.edu/notes/cs229-notes1.pdf" rel="nofollow noreferrer">http://cs229.stanford.edu/notes/cs229-notes1.pdf</a><br />
  This lecturer introduces an alternative approach to gradient descent that is called &quot;Normal Equation&quot; and the equation is as follows:</p>
  <p><span class="math-container">$$\theta=\left(X^{T} X\right)^{-1} X^{T} \vec{y}$$</span></p>
  <p>The normal equation can directly compute the <span class="math-container">$\theta$</span>.</p>
  <p>If the normal equation works, why do we need gradient descent? What is the trade-off between these two methods?</p>
  
Tags: <machine-learning><optimization><gradient-descent>
Title: If the normal equation works, why do we need gradient descent?
UserDisplayName: Cheleeger Ken
ThreadText: # If the normal equation works, why do we need gradient descent?
  
  (Asked by: username_0 on 2020-07-08)
  
  Recently, I followed the open course CS229,
  <http://cs229.stanford.edu/notes/cs229-notes1.pdf>  
  This lecturer introduces an alternative approach to gradient descent that is called "Normal Equation" and the equation is as follows:
  
  $$\theta=\left(X^{T} X\right)^{-1} X^{T} \vec{y}$$
  
  The normal equation can directly compute the $\theta$.
  
  If the normal equation works, why do we need gradient descent? What is the trade-off between these two methods?
  
  **Tags:** machine-learning, optimization, gradient-descent
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-07-08. Score: 1 | Accepted answer
  
  That [normal equation](https://mathworld.wolfram.com/NormalEquation.html) is sometimes called the [closed-form solution](https://mathworld.wolfram.com/Closed-FormSolution.html).
  
  The short answer to your question is that the closed-form solution may be impractical or unavailable in certain cases or the iterative numerical method (such as gradient descent) may be more efficient (in terms of resources).
  
  [This answer](https://stats.stackexchange.com/a/23132/82135) gives you more details and an example.
  
  
  ---
  
  