Answers.Body: <p><strong>Bayes Error Rate</strong></p>
  <p>For the general case of K different classes, the probability of classifing x instance correctly is:</p>
  <p><span class="math-container">\begin{equation} \label{eq1}
  \begin{split}
  P(correct) &amp; = \sum_{i=1}^{K} p(x \in H_i, C_i) \\
   &amp; = \sum_{i=1}^{K} \int_{x \in H_i} p(x,C_i) \, dx\\
   &amp; = \sum_{i=1}^{K} \int_{x \in H_i} P(C_i|x)p(x)\,dx \\
  \end{split}
  \end{equation}</span></p>
  <p>where <span class="math-container">$H_i$</span> is the region where class <span class="math-container">$i$</span> has the highest posterior. So the Bayes Error Rate is:</p>
  <p><span class="math-container">\begin{equation} \label{eq2}
  \begin{split}
  P(error) &amp; = 1 - p(correct) \\
   &amp; = 1 - \sum_{i=1}^{K} \int_{x \in H_i} P(C_i|x)p(x)\,dx
  \end{split}
  \end{equation}</span></p>
  <p>Be careful if we drew <span class="math-container">$x$</span> from <span class="math-container">$H_i$</span>, the region covered/classified by <span class="math-container">$C_i$</span>, <span class="math-container">$P(C_i|x) \ne 1$</span>, because that would mean that we surely predict correctly all the time. If <span class="math-container">$x$</span> belongs to a decision area, this does not imply that it belongs to the corresponding class.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-06-05T03:06:15.880Z
Answers.Text: Ah, I think I get it now. We are looking at the region $H_i$ and integrating the probability that our $x$ would occur (given class $C_i$) over the entire region. Does that sound correct?
Answers.UserDisplayName: EntangledLoops
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-06-04T23:49:33.480Z
Answers.LastActivityDate: 2020-07-03T18:07:46.323Z
Answers.LastEditDate: 2020-07-03T18:07:46.323Z
Answers.UserDisplayName: ddaedalus
Body: <p>My questions concern a particular formulation of the Bayes error rate from <a href="https://en.wikipedia.org/wiki/Bayes_error_rate" rel="nofollow noreferrer">Wikipedia</a>, summarized below.</p>
  <blockquote>
  <p>For a multiclass classifier, the Bayes error rate may be calculated as follows: <span class="math-container">$$p = 1 - \sum_{C_i \ne C_{\max,x}} \int_{x \in H_i} P(C_i|x)p(x)\,dx$$</span> where <span class="math-container">$x$</span> is an instance, <span class="math-container">$C_i$</span> is a class into which an instance is classified, <span class="math-container">$H_i$</span> is the area/region that a classifier function <span class="math-container">$h$</span> classifies as <span class="math-container">$C_i$</span>.</p>
  </blockquote>
  <p>We are interested in the probability of <em>mis</em>classifying an instance, so we wish to sum up the probability of each unlikely class label (hence we want to look at <span class="math-container">$C_i \ne C_{\max, x}$</span>).</p>
  <p>However, the integral is confusing me. We want to integrate an area corresponding to the probability that we choose label <span class="math-container">$C_i$</span> given <span class="math-container">$x$</span>. But we drew <span class="math-container">$x$</span> from <span class="math-container">$H_i$</span>, the region covered/classified by <span class="math-container">$C_i$</span>, so wouldn't <span class="math-container">$P(C_i|x) = 1$</span>?</p>
  <p>I think most of my confusion will be resolved if someone can help clarify the intention of the integral.</p>
  <p>Is it to draw random samples from the total space of <span class="math-container">$h$</span> (the classifier function), and then sum the probabilities from each classified <span class="math-container">$C_i \ne C_{\max, x}$</span>? How does <span class="math-container">$x$</span> exist in the outer summation before it has been sampled from <span class="math-container">$H_i$</span> in the integral?</p>
  
Tags: <classification><probability-theory><bayes-error-rate>
Title: How is the formula for the Bayes error rate with an integral derived?
UserDisplayName: EntangledLoops
ThreadText: # How is the formula for the Bayes error rate with an integral derived?
  
  (Asked by: username_0 on 2020-06-04)
  
  My questions concern a particular formulation of the Bayes error rate from [Wikipedia](https://en.wikipedia.org/wiki/Bayes_error_rate), summarized below.
  
  > For a multiclass classifier, the Bayes error rate may be calculated as follows: $$p = 1 - \sum\_{C\_i \ne C\_{\max,x}} \int\_{x \in H\_i} P(C\_i|x)p(x)\,dx$$ where $x$ is an instance, $C\_i$ is a class into which an instance is classified, $H\_i$ is the area/region that a classifier function $h$ classifies as $C\_i$.
  
  We are interested in the probability of *mis*classifying an instance, so we wish to sum up the probability of each unlikely class label (hence we want to look at $C\_i \ne C\_{\max, x}$).
  
  However, the integral is confusing me. We want to integrate an area corresponding to the probability that we choose label $C\_i$ given $x$. But we drew $x$ from $H\_i$, the region covered/classified by $C\_i$, so wouldn't $P(C\_i|x) = 1$?
  
  I think most of my confusion will be resolved if someone can help clarify the intention of the integral.
  
  Is it to draw random samples from the total space of $h$ (the classifier function), and then sum the probabilities from each classified $C\_i \ne C\_{\max, x}$? How does $x$ exist in the outer summation before it has been sampled from $H\_i$ in the integral?
  
  **Tags:** classification, probability-theory, bayes-error-rate
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-06-04. Score: 3 | Accepted answer
  
  **Bayes Error Rate**
  
  For the general case of K different classes, the probability of classifing x instance correctly is:
  
  \begin{equation} \label{eq1}
  \begin{split}
  P(correct) & = \sum\_{i=1}^{K} p(x \in H\_i, C\_i) \\
  & = \sum\_{i=1}^{K} \int\_{x \in H\_i} p(x,C\_i) \, dx\\
  & = \sum\_{i=1}^{K} \int\_{x \in H\_i} P(C\_i|x)p(x)\,dx \\
  \end{split}
  \end{equation}
  
  where $H\_i$ is the region where class $i$ has the highest posterior. So the Bayes Error Rate is:
  
  \begin{equation} \label{eq2}
  \begin{split}
  P(error) & = 1 - p(correct) \\
  & = 1 - \sum\_{i=1}^{K} \int\_{x \in H\_i} P(C\_i|x)p(x)\,dx
  \end{split}
  \end{equation}
  
  Be careful if we drew $x$ from $H\_i$, the region covered/classified by $C\_i$, $P(C\_i|x) \ne 1$, because that would mean that we surely predict correctly all the time. If $x$ belongs to a decision area, this does not imply that it belongs to the corresponding class.
  
  > **Comments:**
  > **username_0**: Ah, I think I get it now. We are looking at the region $H_i$ and integrating the probability that our $x$ would occur (given class $C_i$) over the entire region. Does that sound correct?  
  
  ---
  
  