Answers.Body: <p>As in the second line, the first two terms are <span class="math-container">$\mathbb{E}_{\hat{y}}$</span>, it means the variable of the expectation is <span class="math-container">$\hat{y}_i$</span> and you can take out <span class="math-container">$y_i$</span>s from <span class="math-container">$\mathbb{E}_{\hat{y}}$</span>s. Now we can use <span class="math-container">$\mathbb{E}\{\hat{y}_i\} = y_i$</span> and rewrite the terms of the second line likes the following:</p>
  <p><span class="math-container">$$
  \mathbb{E}_{\hat{y}}(\sum_i y_i)^2 = (\sum_i y_i)^2 \\
  \mathbb{E}_{\hat{y}}\left[(\sum_i y_i)(\sum_i \hat{y}_i)\right] = (\sum_i y_i) \mathbb{E}_{\hat{y}}(\sum_i \hat{y}_i) = (\sum_i y_i)^2
  $$</span></p>
  <p>The second line is written based on linearity of the expectation and <span class="math-container">$\mathbb{E}\{\hat{y}_i\} = y_i$</span>. Hence, we can rewrite the second line like the following as <span class="math-container">$\sum_i y_i = \mathbb{E}_{\hat{y}}(\sum_i \hat{y}_i)$</span>:</p>
  <p><span class="math-container">$$
  \frac{1}{N^2}\left[\mathbb{E}_{\hat{y}}(\sum_i \hat{y}_i)^2 - \left(\mathbb{E}_{\hat{y}}(\sum_i \hat{y}_i)\right)^2\right]
  $$</span></p>
  <p>And the final step is using this formula <span class="math-container">$Var(X) = E(X^2) - (E(X))^2$</span>  and take <span class="math-container">$X = \sum_i \hat{y}_i$</span>:</p>
  <p><span class="math-container">$$
  \frac{1}{N^2}\left[\mathbb{E}_{\hat{y}}(\sum_i \hat{y}_i)^2 - \left(\mathbb{E}_{\hat{y}}(\sum_i \hat{y}_i)\right)^2\right] = \frac{1}{N^2} Var(\sum_i \hat{y}_i)
  $$</span></p>
  <p>From variance to covariance, you can use <a href="https://en.wikipedia.org/wiki/Variance" rel="nofollow noreferrer">this formula</a>: <span class="math-container">$$Var(\sum_{i=1}^nX_i) = \sum_{i=1}^n\sum_{j=1}^n cov(X_i, X_j)$$</span></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-11-30T15:05:50.437Z
Answers.LastActivityDate: 2020-11-30T15:12:27.343Z
Answers.LastEditDate: 2020-11-30T15:12:27.343Z
Answers.UserDisplayName: OmG
Body: <p>I am trying to understand the last two lines of this math notation (from <a href="https://arxiv.org/pdf/1803.04189.pdf" rel="nofollow noreferrer">this paper</a>).</p>
  <p><a href="https://i.sstatic.net/OgQGk.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/OgQGk.jpg" alt="enter image description here" /></a></p>
  <p>How did the Var and double summation of the Cov come to the equation?</p>
  <p>The first two lines I understood something like <span class="math-container">$(a-b)^2 = a^2 -2ab +b^2$</span>.</p>
  
Tags: <machine-learning><papers><objective-functions><math>
Title: How did the variance and double summation of the covariance come to the L2 minimization equation?
UserDisplayName: Markov
ThreadText: # How did the variance and double summation of the covariance come to the L2 minimization equation?
  
  (Asked by: username_0 on 2020-11-30)
  
  I am trying to understand the last two lines of this math notation (from [this paper](https://arxiv.org/pdf/1803.04189.pdf)).
  
  [![enter image description here](https://i.sstatic.net/OgQGk.jpg)](https://i.sstatic.net/OgQGk.jpg)
  
  How did the Var and double summation of the Cov come to the equation?
  
  The first two lines I understood something like $(a-b)^2 = a^2 -2ab +b^2$.
  
  **Tags:** machine-learning, papers, objective-functions, math
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-11-30. Score: 1
  
  As in the second line, the first two terms are $\mathbb{E}\_{\hat{y}}$, it means the variable of the expectation is $\hat{y}\_i$ and you can take out $y\_i$s from $\mathbb{E}\_{\hat{y}}$s. Now we can use $\mathbb{E}\{\hat{y}\_i\} = y\_i$ and rewrite the terms of the second line likes the following:
  
  $$
  \mathbb{E}\_{\hat{y}}(\sum\_i y\_i)^2 = (\sum\_i y\_i)^2 \\
  \mathbb{E}\_{\hat{y}}\left[(\sum\_i y\_i)(\sum\_i \hat{y}\_i)\right] = (\sum\_i y\_i) \mathbb{E}\_{\hat{y}}(\sum\_i \hat{y}\_i) = (\sum\_i y\_i)^2
  $$
  
  The second line is written based on linearity of the expectation and $\mathbb{E}\{\hat{y}\_i\} = y\_i$. Hence, we can rewrite the second line like the following as $\sum\_i y\_i = \mathbb{E}\_{\hat{y}}(\sum\_i \hat{y}\_i)$:
  
  $$
  \frac{1}{N^2}\left[\mathbb{E}\_{\hat{y}}(\sum\_i \hat{y}\_i)^2 - \left(\mathbb{E}\_{\hat{y}}(\sum\_i \hat{y}\_i)\right)^2\right]
  $$
  
  And the final step is using this formula $Var(X) = E(X^2) - (E(X))^2$ and take $X = \sum\_i \hat{y}\_i$:
  
  $$
  \frac{1}{N^2}\left[\mathbb{E}\_{\hat{y}}(\sum\_i \hat{y}\_i)^2 - \left(\mathbb{E}\_{\hat{y}}(\sum\_i \hat{y}\_i)\right)^2\right] = \frac{1}{N^2} Var(\sum\_i \hat{y}\_i)
  $$
  
  From variance to covariance, you can use [this formula](https://en.wikipedia.org/wiki/Variance): $$Var(\sum\_{i=1}^nX\_i) = \sum\_{i=1}^n\sum\_{j=1}^n cov(X\_i, X\_j)$$
  
  
  ---
  
  