Answers.Body: <p>The misclassifications that <em>could arise if <span class="math-container">$\hat{x}$</span> is used as decision boundary</em> are:</p>
  
  <p>a) Classifying a point as <span class="math-container">$C_2$</span> when actually it was <span class="math-container">$C_1$</span> -- which will only happen when <span class="math-container">$x &gt; \hat{x}$</span> as only the points greater than <span class="math-container">$\hat{x}$</span> are being classified as <span class="math-container">$C_2$</span>.</p>
  
  <p>b) Classifying a point as <span class="math-container">$C_1$</span> when actually it was <span class="math-container">$C_2$</span> -- which will only happen when <span class="math-container">$x &lt; \hat{x}$</span> as only the points less than <span class="math-container">$\hat{x}$</span> are being classified to be from class <span class="math-container">$C_1$</span>.</p>
  
  <blockquote>
    <p>2) Why is the green area with x &lt; x0 a misclassification? </p>
  </blockquote>
  
  <p>If we label a point -- the <span class="math-container">$x$</span> which is from the interval on the horizontal axis which corresponds to the <a href="https://en.wikipedia.org/wiki/Probability" rel="nofollow noreferrer">green area -- which is a probability</a> as being from either class then there is a good chance that it could be misclassified. This is because both the <a href="https://en.wikipedia.org/wiki/Joint_probability_distribution" rel="nofollow noreferrer">joint distributions'</a> curves are above the axis for that interval on the axis and have some area (probability) for that interval on the horizontal axis.</p>
  
  <p>There is a positive probability that a point that was drawn from the interval (a subset of the sample space, an event), which corresponds to the green area, was drawn from either of the two joint distributions or belongs to either of the two classes <span class="math-container">$C_1$</span> and <span class="math-container">$C_2$</span>. The probability is the area under the curve.</p>
  
  <blockquote>
    <p>It's classified as C1 and it is supposed to be C1 right?</p>
  </blockquote>
  
  <p>The same data point could well be generated from multiple different probability distributions. Here, you have an illustration of two probability distributions for which the author is trying to find the optimal decision boundary that will minimize the misclassification "region".</p>
  
  <p>The point is not supposed to be from <span class="math-container">$C_1$</span>, even if the <a href="https://en.wikipedia.org/wiki/Probability_density_function" rel="nofollow noreferrer">density of the joint prob. distribution</a> of <span class="math-container">$(x, C_1)$</span> has <a href="https://math.stackexchange.com/questions/210630/what-does-the-value-of-a-probability-density-function-pdf-at-some-x-indicate">higher values</a>. It could well be from <span class="math-container">$C_2$</span>. And, when one classifies the same data point as being from class <span class="math-container">$C_1$</span> and make an error. This is why the whole area under the curve <span class="math-container">$p(x, C_2)$</span> has been painted green -- which means that there is some probability that a point could be from the distribution <span class="math-container">$p(x, C_2)$</span> and if we blindly label all the points to be from <span class="math-container">$C_1$</span> will certainly lead to some misclassifications. </p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-08-07T18:08:11.180Z
Answers.LastActivityDate: 2019-08-09T06:28:41.967Z
Answers.LastEditDate: 2019-08-09T06:28:41.967Z
Answers.UserDisplayName: naive
Body: <p>In the image given below, I do not understand a few things</p>
  
  <p>1) Why is an entire area colored to signify misclassification? For the given decision boundary, only the points between <span class="math-container">$x_0$</span> and the decision boundary signify misclassification right? It's supposed to be only a set of points on the x-axis, not an area.</p>
  
  <p>2) Why is the green area with <span class="math-container">$x &lt; x_0$</span> a misclassification? It's classified as <span class="math-container">$C_1$</span> and it is supposed to be <span class="math-container">$C_1$</span> right?</p>
  
  <p>3) Similarly, why is the blue area a misclassification? Any <span class="math-container">$x &gt;$</span> the decision boundary belongs to <span class="math-container">$C_2$</span> and is also classified as such...</p>
  
  <p><a href="https://i.sstatic.net/THTLv.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/THTLv.png" alt="enter image description here"></a></p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-08-18T15:29:01.157Z
Comments.Text: It would be interesting to know your views on the answer and help improve it, if needed.
Comments.UserDisplayName: naive
Tags: <machine-learning><probability><decision-theory><probability-distribution>
Title: Why is the entire area of a join probability distribution considered when it comes to calculating misclassification?
UserDisplayName: sage76
ThreadText: # Why is the entire area of a join probability distribution considered when it comes to calculating misclassification?
  
  (Asked by: username_0 on 2019-08-06)
  
  In the image given below, I do not understand a few things
  
  1) Why is an entire area colored to signify misclassification? For the given decision boundary, only the points between $x\_0$ and the decision boundary signify misclassification right? It's supposed to be only a set of points on the x-axis, not an area.
  
  2) Why is the green area with $x < x\_0$ a misclassification? It's classified as $C\_1$ and it is supposed to be $C\_1$ right?
  
  3) Similarly, why is the blue area a misclassification? Any $x >$ the decision boundary belongs to $C\_2$ and is also classified as such...
  
  [![enter image description here](https://i.sstatic.net/THTLv.png)](https://i.sstatic.net/THTLv.png)
  
  **Tags:** machine-learning, probability, decision-theory, probability-distribution
  
  > **Comments:**
  > **username_1**: It would be interesting to know your views on the answer and help improve it, if needed.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-08-07. Score: 0 | Accepted answer
  
  The misclassifications that *could arise if $\hat{x}$ is used as decision boundary* are:
  
  a) Classifying a point as $C\_2$ when actually it was $C\_1$ -- which will only happen when $x > \hat{x}$ as only the points greater than $\hat{x}$ are being classified as $C\_2$.
  
  b) Classifying a point as $C\_1$ when actually it was $C\_2$ -- which will only happen when $x < \hat{x}$ as only the points less than $\hat{x}$ are being classified to be from class $C\_1$.
  
  > 2) Why is the green area with x < x0 a misclassification?
  
  If we label a point -- the $x$ which is from the interval on the horizontal axis which corresponds to the [green area -- which is a probability](https://en.wikipedia.org/wiki/Probability) as being from either class then there is a good chance that it could be misclassified. This is because both the [joint distributions'](https://en.wikipedia.org/wiki/Joint_probability_distribution) curves are above the axis for that interval on the axis and have some area (probability) for that interval on the horizontal axis.
  
  There is a positive probability that a point that was drawn from the interval (a subset of the sample space, an event), which corresponds to the green area, was drawn from either of the two joint distributions or belongs to either of the two classes $C\_1$ and $C\_2$. The probability is the area under the curve.
  
  > It's classified as C1 and it is supposed to be C1 right?
  
  The same data point could well be generated from multiple different probability distributions. Here, you have an illustration of two probability distributions for which the author is trying to find the optimal decision boundary that will minimize the misclassification "region".
  
  The point is not supposed to be from $C\_1$, even if the [density of the joint prob. distribution](https://en.wikipedia.org/wiki/Probability_density_function) of $(x, C\_1)$ has [higher values](https://math.stackexchange.com/questions/210630/what-does-the-value-of-a-probability-density-function-pdf-at-some-x-indicate). It could well be from $C\_2$. And, when one classifies the same data point as being from class $C\_1$ and make an error. This is why the whole area under the curve $p(x, C\_2)$ has been painted green -- which means that there is some probability that a point could be from the distribution $p(x, C\_2)$ and if we blindly label all the points to be from $C\_1$ will certainly lead to some misclassifications.
  
  
  ---
  
  