Answers.Body: <p>Both risk formulations here are for supervised learning of a target label in the typical form of <span class="math-container">$\hat{y}=f(\mathbf{x})$</span>, but they're not the usual empirical risk. Especially from your second version, it's an expectation thus it's the true risk assuming a known data (in the generative pair form <span class="math-container">$(\mathbf{x},y)$</span>) generating full joint distribution <span class="math-container">$p(\mathbf{x},y)$</span>. Therefore you have to integrate over both <span class="math-container">$\mathbf{x}$</span> and <span class="math-container">$y$</span>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-07-17T08:01:05.787Z
Answers.LastActivityDate: 2024-07-17T08:01:05.787Z
Answers.UserDisplayName: cinch
Body: <p>The formula for Generalization error, taken from Wikipedia:</p>
  <p><span class="math-container">$$
  I[f]=\int _{X\times Y}V(f({\vec {x}}),y)\rho ({\vec {x}},y)d{\vec {x}}dy
  $$</span></p>
  <p>d2l.ai's version:</p>
  <p><span class="math-container">$$
   R[p, f] = E_{(\mathbf{x}, y) \sim P} [l(\mathbf{x}, y, f(\mathbf{x}))] =
     \int \int l(\mathbf{x}, y, f(\mathbf{x})) p(\mathbf{x}, y) \;d\mathbf{x} dy
  $$</span></p>
  <p>Compared to the discrete Empirical one that is simply the sum of all data points, why does the integral is also over Y, when X is enough to &quot;cover&quot; every cases.</p>
  <p>(Forgive me for any incorrect uses of terms.)</p>
  
Tags: <generalization>
Title: Why does generalization error's integral integrate over both X and Y, not just X?
UserDisplayName: Daanturo
ThreadText: # Why does generalization error's integral integrate over both X and Y, not just X?
  
  (Asked by: username_0 on 2024-07-17)
  
  The formula for Generalization error, taken from Wikipedia:
  
  $$
  I[f]=\int \_{X\times Y}V(f({\vec {x}}),y)\rho ({\vec {x}},y)d{\vec {x}}dy
  $$
  
  d2l.ai's version:
  
  $$
  R[p, f] = E\_{(\mathbf{x}, y) \sim P} [l(\mathbf{x}, y, f(\mathbf{x}))] =
  \int \int l(\mathbf{x}, y, f(\mathbf{x})) p(\mathbf{x}, y) \;d\mathbf{x} dy
  $$
  
  Compared to the discrete Empirical one that is simply the sum of all data points, why does the integral is also over Y, when X is enough to "cover" every cases.
  
  (Forgive me for any incorrect uses of terms.)
  
  **Tags:** generalization
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2024-07-17. Score: -1 | Accepted answer
  
  Both risk formulations here are for supervised learning of a target label in the typical form of $\hat{y}=f(\mathbf{x})$, but they're not the usual empirical risk. Especially from your second version, it's an expectation thus it's the true risk assuming a known data (in the generative pair form $(\mathbf{x},y)$) generating full joint distribution $p(\mathbf{x},y)$. Therefore you have to integrate over both $\mathbf{x}$ and $y$.
  
  
  ---
  
  