Body: <p>The following derivation is taken from  <a href="https://www.deeplearningbook.org/contents/ml.html" rel="nofollow noreferrer">Chapter 5: Machine Learning Basics</a> from the book titled <a href="https://www.deeplearningbook.org/" rel="nofollow noreferrer">Deep Learning</a> (by Aaron Courville et al.)</p>
  <p>I am facing difficulty in understanding the zero derivative related to minimizing gradient of mean squared error <span class="math-container">$\nabla_w  \text{MSE}_{train} = 0$</span></p>
  <blockquote>
  <p><span class="math-container">$\nabla_w \text{MSE}_{train} = 0$</span></p>
  <p><span class="math-container">$\implies \nabla_w(Xw - y)^T(Xw-y) = 0$</span></p>
  <p><span class="math-container">$\implies \nabla_w(w^T X^T-
   y^T)(Xw-y) = 0$</span></p>
  <p><span class="math-container">$\implies \nabla_w(w^T X^T-
   y^T)(Xw-y) = 0$</span></p>
  <p><span class="math-container">$\implies \nabla_w(w^T X^TXw - w^T
   X^Ty
   -y^TXw+y^Ty) = 0$</span></p>
  <p><span class="math-container">$\implies \nabla_w(w^T X^TXw - 2 w^T
   X^Ty +y^Ty) = 0$</span></p>
  <p><span class="math-container">$\implies 2 X^TXw - 2 X^Ty = 0$</span></p>
  <p><span class="math-container">$\implies w = 2 (X^TX)^{-1} X^Ty = 0$</span></p>
  </blockquote>
  <p>I have had difficulty in understanding the flow of the following two lines</p>
  <blockquote>
  <p><span class="math-container">$\implies \nabla_w(w^T X^TXw - w^T
   X^Ty
   -y^TXw+y^Ty) = 0$</span></p>
  <p><span class="math-container">$\implies \nabla_w(w^T X^TXw - 2 w^T
   X^Ty +y^Ty) = 0$</span></p>
  </blockquote>
  <p>The first doubt is about in the first two lines, it is possible only if (which I feel is untrue)</p>
  <p><span class="math-container">$w^T X^Ty = y^TXw$</span></p>
  <p>Note: <span class="math-container">$X,y$</span> here refers to input and outputs of train data</p>
  
Tags: <math><gradient><matrices>
Title: Isssue in understanding the derivation regarding mean squared error
UserDisplayName: hanugm
ThreadText: # Isssue in understanding the derivation regarding mean squared error
  
  (Asked by: username_0 on 2021-08-29)
  
  The following derivation is taken from [Chapter 5: Machine Learning Basics](https://www.deeplearningbook.org/contents/ml.html) from the book titled [Deep Learning](https://www.deeplearningbook.org/) (by Aaron Courville et al.)
  
  I am facing difficulty in understanding the zero derivative related to minimizing gradient of mean squared error $\nabla\_w \text{MSE}\_{train} = 0$
  
  > $\nabla\_w \text{MSE}\_{train} = 0$
  >
  > $\implies \nabla\_w(Xw - y)^T(Xw-y) = 0$
  >
  > $\implies \nabla\_w(w^T X^T-
  > y^T)(Xw-y) = 0$
  >
  > $\implies \nabla\_w(w^T X^T-
  > y^T)(Xw-y) = 0$
  >
  > $\implies \nabla\_w(w^T X^TXw - w^T
  > X^Ty
  > -y^TXw+y^Ty) = 0$
  >
  > $\implies \nabla\_w(w^T X^TXw - 2 w^T
  > X^Ty +y^Ty) = 0$
  >
  > $\implies 2 X^TXw - 2 X^Ty = 0$
  >
  > $\implies w = 2 (X^TX)^{-1} X^Ty = 0$
  
  I have had difficulty in understanding the flow of the following two lines
  
  > $\implies \nabla\_w(w^T X^TXw - w^T
  > X^Ty
  > -y^TXw+y^Ty) = 0$
  >
  > $\implies \nabla\_w(w^T X^TXw - 2 w^T
  > X^Ty +y^Ty) = 0$
  
  The first doubt is about in the first two lines, it is possible only if (which I feel is untrue)
  
  $w^T X^Ty = y^TXw$
  
  Note: $X,y$ here refers to input and outputs of train data
  
  **Tags:** math, gradient, matrices
  
  ---
  
  ## 0 answers
  
  