Answers.Body: <p>This is very easy to prove.</p>
  <p>Let's first prove that, if <span class="math-container">$\hat{y}_k = y_k$</span>, then the <span class="math-container">$E = 0$</span>. I will leave all steps, so that it's super clear.</p>
  <p><span class="math-container">\begin{align}
  E
  &amp;=\frac{1}{2}\sum_k(\hat{y}_k - y_k)^2 \\
  &amp;=\frac{1}{2}\sum_k(y_k - y_k)^2\\
  &amp;=\frac{1}{2}\sum_k(0)^2\\
  &amp;=\frac{1}{2}\sum_k 0\\
  &amp;=\frac{1}{2} 0\\
  &amp;=0\\
  \end{align}</span></p>
  <p>To prove the other way around, i.e. if <span class="math-container">$E = 0$</span>, then <span class="math-container">$\hat{y}_k = y_k$</span>, you can do as follows</p>
  <p><span class="math-container">\begin{align}
  \frac{1}{2}\sum_k(\hat{y}_k - y_k)^2 
  &amp;=E\\
  &amp;=0
  \end{align}</span>
  Recall now that any number squared is non-negative (i.e. positive or zero). Given that <span class="math-container">$(\hat{y}_k - y_k)^2 $</span> is non-negative, then <span class="math-container">$\sum_k(\hat{y}_k - y_k)^2$</span> is a sum of non-negative numbers. The only way that a sum of non-negative numbers is equal to zero is if all numbers are zero, so we must have <span class="math-container">$\hat{y}_k = y_k$</span> (because any non-zero number squared is non-zero).</p>
  <p>(Note that <span class="math-container">$E$</span> is the <strong>mean squared error</strong>, i.e. a loss function, and it's <strong>not</strong> the back-propagation algorithm, which is just the algorithm that you use to compute partial derivatives of <span class="math-container">$E$</span> with respect to the parameters of the model, which are not even visible in the way you wrote <span class="math-container">$E$</span>).</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-12-24T12:42:12.133Z
Answers.LastActivityDate: 2020-12-24T12:59:00.723Z
Answers.LastEditDate: 2020-12-24T12:59:00.723Z
Answers.UserDisplayName: nbro
Body: <p>In the back-propogation algorithm, the error term is:</p>
  <p><span class="math-container">$$
  E=\frac{1}{2}\sum_k(\hat{y}_k - y_k)^2,
  $$</span></p>
  <p>where <span class="math-container">$\hat{y}_k$</span> is a vector of outputs from the network, <span class="math-container">$y_k$</span> is the vector of correct labels (and we work out the error by calculating predicted-observed, squaring the answer, and then summing the answers for each <span class="math-container">$k$</span> (and dividing by 2).</p>
  <p>How do you prove that if this answer is <span class="math-container">$0$</span> (i.e., if <span class="math-container">$E=0$</span>), then <span class="math-container">$\hat{y}_k=y_k$</span> for all <span class="math-container">$k$</span>?</p>
  
Tags: <deep-learning><proofs><mean-squared-error>
Title: How do I prove that the MSE is zero when all predictions are equal to the corresponding labels?
UserDisplayName: Slowat_Kela
ThreadText: # How do I prove that the MSE is zero when all predictions are equal to the corresponding labels?
  
  (Asked by: username_0 on 2020-12-18)
  
  In the back-propogation algorithm, the error term is:
  
  $$
  E=\frac{1}{2}\sum\_k(\hat{y}\_k - y\_k)^2,
  $$
  
  where $\hat{y}\_k$ is a vector of outputs from the network, $y\_k$ is the vector of correct labels (and we work out the error by calculating predicted-observed, squaring the answer, and then summing the answers for each $k$ (and dividing by 2).
  
  How do you prove that if this answer is $0$ (i.e., if $E=0$), then $\hat{y}\_k=y\_k$ for all $k$?
  
  **Tags:** deep-learning, proofs, mean-squared-error
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-12-24. Score: 2 | Accepted answer
  
  This is very easy to prove.
  
  Let's first prove that, if $\hat{y}\_k = y\_k$, then the $E = 0$. I will leave all steps, so that it's super clear.
  
  \begin{align}
  E
  &=\frac{1}{2}\sum\_k(\hat{y}\_k - y\_k)^2 \\
  &=\frac{1}{2}\sum\_k(y\_k - y\_k)^2\\
  &=\frac{1}{2}\sum\_k(0)^2\\
  &=\frac{1}{2}\sum\_k 0\\
  &=\frac{1}{2} 0\\
  &=0\\
  \end{align}
  
  To prove the other way around, i.e. if $E = 0$, then $\hat{y}\_k = y\_k$, you can do as follows
  
  \begin{align}
  \frac{1}{2}\sum\_k(\hat{y}\_k - y\_k)^2
  &=E\\
  &=0
  \end{align}
  Recall now that any number squared is non-negative (i.e. positive or zero). Given that $(\hat{y}\_k - y\_k)^2 $ is non-negative, then $\sum\_k(\hat{y}\_k - y\_k)^2$ is a sum of non-negative numbers. The only way that a sum of non-negative numbers is equal to zero is if all numbers are zero, so we must have $\hat{y}\_k = y\_k$ (because any non-zero number squared is non-zero).
  
  (Note that $E$ is the **mean squared error**, i.e. a loss function, and it's **not** the back-propagation algorithm, which is just the algorithm that you use to compute partial derivatives of $E$ with respect to the parameters of the model, which are not even visible in the way you wrote $E$).
  
  
  ---
  
  