Answers.Body: <p>This is the definition of conditional probability + Total probability decomposition formula:</p>
  
  <p><span class="math-container">$p(y|x) = \frac{p(y,x}{p(x)} = \frac{p(x,y)}{\sum_{y'}p(x,y')}$</span>. </p>
  
  <p>The idea is to use some unsupervised learning algorithm to learn the distribution <span class="math-container">$p(x,y)$</span> for every possible value of <span class="math-container">$y$</span>, and by using the previous formula you can find <span class="math-container">$p(y|x)$</span>. </p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-02-25T11:05:39.137Z
Answers.Text: Thanks for the answer. What is the "previous" formula you're referring to?
Answers.UserDisplayName: The Pointer
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-02-25T11:15:26.000Z
Answers.Text: I pionted to p(y|x)=p(y,xp(x)=p(x,y)∑y′p(x,y′)
Answers.UserDisplayName: hola
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-02-25T11:20:39.277Z
Answers.Text: Ok, thanks. What is $y'$ supposed to be, as opposed to $y$?
Answers.UserDisplayName: The Pointer
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-02-25T11:37:31.787Z
Answers.Text: when we write a sum over y', it means that y' is ranging over all possible values of y. Consider for example the case where you have a binary classification, then y' can take values 0 and 1. In that case p(y=0|x) = p(x,y=0)/ [p(x,y=0) + p(x,y=1)]
Answers.UserDisplayName: hola
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-02-25T10:50:03.083Z
Answers.LastActivityDate: 2020-02-25T10:58:48.963Z
Answers.LastEditDate: 2020-02-25T10:58:48.963Z
Answers.UserDisplayName: hola
Body: <p>I am currently studying Deep Learning by Goodfellow, Bengio, and Courville. In chapter <strong>5.1.2 The Performance Measure,</strong> <span class="math-container">$P$</span>, the authors say the following:</p>
  <blockquote>
  <p>Unsupervised learning and supervised learning are not formally defined terms. The lines between them are often blurred. Many machine learning technologies can  be used to perform both tasks. For example, the chain rule of probability states that for a vector <span class="math-container">$\mathbf{x} \in \mathbb{R}^n$</span>, the joint distribution can be decomposed as</p>
  <p><span class="math-container">$$p(\mathbf{x}) = \prod_{i = 1}^n p(x_i \vert x_1, \dots, x_{i - 1} ).$$</span></p>
  <p>This decomposition means that we can solve the ostensibly unsupervised problem of modeling <span class="math-container">$p(\mathbf{x})$</span> by splitting it into <span class="math-container">$n$</span> supervised learning problems. Alternatively, we  can solve the supervised learning problem of learning <span class="math-container">$p(y \vert \mathbf{x})$</span> by using traditional unsupervised technologies to learn the joint distribution <span class="math-container">$p(\mathbf{x}, y)$</span>, then inferring</p>
  <p><span class="math-container">$$p(y \vert \mathbf{x} ) = \dfrac{p(\mathbf{x}, y)}{\sum_{y'}p(\mathbf{x}, y')}.$$</span></p>
  </blockquote>
  <p>I found this part vague:</p>
  <blockquote>
  <p>Alternatively, we can solve the supervised learning problem of learning <span class="math-container">$p(y \vert \mathbf{x})$</span> by using traditional unsupervised technologies to learn the joint distribution <span class="math-container">$p(\mathbf{x}, y)$</span>, then inferring</p>
  <p><span class="math-container">$$p(y \vert \mathbf{x} ) = \dfrac{p(\mathbf{x}, y)}{\sum_{y'}p(\mathbf{x}, y')}.$$</span></p>
  </blockquote>
  <p>Can someone please elaborate on this, and also explain more clearly the role of <span class="math-container">$p(y \vert \mathbf{x} ) = \dfrac{p(\mathbf{x}, y)}{\sum_{y'}p(\mathbf{x}, y')}$</span>?</p>
  <p>I would greatly appreciate it if people would please take the time to clarify this.</p>
  
Tags: <unsupervised-learning><supervised-learning><probability-distribution><conditional-probability>
Title: Solving the supervised learning problem of learning $p(y \vert \mathbf{x})$ by using traditional unsupervised technologies to learn $p(\mathbf{x}, y)$
UserDisplayName: The Pointer
ThreadText: # Solving the supervised learning problem of learning $p(y \vert \mathbf{x})$ by using traditional unsupervised technologies to learn $p(\mathbf{x}, y)$
  
  (Asked by: username_0 on 2020-02-25)
  
  I am currently studying Deep Learning by Goodfellow, Bengio, and Courville. In chapter **5.1.2 The Performance Measure,** $P$, the authors say the following:
  
  > Unsupervised learning and supervised learning are not formally defined terms. The lines between them are often blurred. Many machine learning technologies can be used to perform both tasks. For example, the chain rule of probability states that for a vector $\mathbf{x} \in \mathbb{R}^n$, the joint distribution can be decomposed as
  >
  > $$p(\mathbf{x}) = \prod\_{i = 1}^n p(x\_i \vert x\_1, \dots, x\_{i - 1} ).$$
  >
  > This decomposition means that we can solve the ostensibly unsupervised problem of modeling $p(\mathbf{x})$ by splitting it into $n$ supervised learning problems. Alternatively, we can solve the supervised learning problem of learning $p(y \vert \mathbf{x})$ by using traditional unsupervised technologies to learn the joint distribution $p(\mathbf{x}, y)$, then inferring
  >
  > $$p(y \vert \mathbf{x} ) = \dfrac{p(\mathbf{x}, y)}{\sum\_{y'}p(\mathbf{x}, y')}.$$
  
  I found this part vague:
  
  > Alternatively, we can solve the supervised learning problem of learning $p(y \vert \mathbf{x})$ by using traditional unsupervised technologies to learn the joint distribution $p(\mathbf{x}, y)$, then inferring
  >
  > $$p(y \vert \mathbf{x} ) = \dfrac{p(\mathbf{x}, y)}{\sum\_{y'}p(\mathbf{x}, y')}.$$
  
  Can someone please elaborate on this, and also explain more clearly the role of $p(y \vert \mathbf{x} ) = \dfrac{p(\mathbf{x}, y)}{\sum\_{y'}p(\mathbf{x}, y')}$?
  
  I would greatly appreciate it if people would please take the time to clarify this.
  
  **Tags:** unsupervised-learning, supervised-learning, probability-distribution, conditional-probability
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-02-25. Score: 2 | Accepted answer
  
  This is the definition of conditional probability + Total probability decomposition formula:
  
  $p(y|x) = \frac{p(y,x}{p(x)} = \frac{p(x,y)}{\sum\_{y'}p(x,y')}$.
  
  The idea is to use some unsupervised learning algorithm to learn the distribution $p(x,y)$ for every possible value of $y$, and by using the previous formula you can find $p(y|x)$.
  
  > **Comments:**
  > **username_0**: Thanks for the answer. What is the "previous" formula you're referring to?  
  > **username_1**: I pionted to p(y|x)=p(y,xp(x)=p(x,y)∑y′p(x,y′)  
  > **username_0**: Ok, thanks. What is $y'$ supposed to be, as opposed to $y$?  
  > **username_1**: when we write a sum over y', it means that y' is ranging over all possible values of y. Consider for example the case where you have a binary classification, then y' can take values 0 and 1. In that case p(y=0|x) = p(x,y=0)/ [p(x,y=0) + p(x,y=1)]  
  
  ---
  
  