Answers.Body: <p><strong>For step 1</strong>: you're on the right track, but the derivation is easier if we leave everything conditional on <span class="math-container">$X$</span>
  <span class="math-container">$$p(\beta| X,y) = \frac{p(y| X, \beta) p(\beta | X)}{p(y |X)}$$</span>
  Now, the two key steps. First, <span class="math-container">$p(\beta | X) = p(\beta)$</span> if <span class="math-container">$X$</span> and <span class="math-container">$\beta$</span> are independent (why is this the case?). Second, <span class="math-container">$p(y |X)$</span> is constant in <span class="math-container">$\beta$</span>. How does multiplying a function by a constant affect the location of its maximum?</p>
  <p><strong>For step 2</strong>: simply substitute the expressions given for <span class="math-container">$p(y|X, \beta)$</span> and <span class="math-container">$p(\beta)$</span>. There will be additional terms that don't depend on <span class="math-container">$\beta$</span>. How does adding a constant to a function affect the location of its maximum?</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-03-09T00:55:48.933Z
Answers.Text: How do we get the first equality of leaving everything conditional on $X$? I don't understand what leaving everything conditional on $X$ means. Also, how do we know that $X$ and $\beta$ are independent?
Answers.UserDisplayName: tesio
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-03-09T01:09:56.073Z
Answers.Text: The idea is that we can apply Bayes rule to conditional distributions that are all conditioned on the same thing. In other words, if we ignored the $X$ conditioning in all terms of the above equation, we would get Bayes rule. It's also valid with the conditioning there. You can prove this by fiddling around with the definitions of conditional distributions. In terms of independence, should we expect the slope of a linear function to depend on the points that we probe it?
Answers.UserDisplayName: Luke
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-03-09T01:31:58.590Z
Answers.Text: I understand the conditional part. For the independence, do you mean that the slope doesn't change if the X changes? Then, the slope depends on $y$ but not $X$?
Answers.UserDisplayName: tesio
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-03-09T17:30:08.867Z
Answers.Text: Yes indeed.  For linear regression, the model that we are fitting is:
  $X  \rightarrow y \leftarrow \beta$
    
  So $X$ and $\beta$ are marginally independent. Note that in standard linear regression,  $X$ is not even a random variable, but rather is assumed to be known exactly
Answers.UserDisplayName: Luke
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-03-09T00:32:55.933Z
Answers.LastActivityDate: 2023-03-09T00:32:55.933Z
Answers.UserDisplayName: Luke
Body: <p>I am taking ML course and I am confused about some derivations of math</p>
  <p><a href="https://i.sstatic.net/ZhfSK.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/ZhfSK.png" alt="enter image description here" /></a>
  <a href="https://i.sstatic.net/BBn0b.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/BBn0b.png" alt="enter image description here" /></a></p>
  <p>Could you explain the two steps I marked on the slides? For the first step, I thought <span class="math-container">$P(beta|X,y) = \frac{P(X,y|beta)P(beta)}{P(X,y)}$</span> but I don't know the further steps to reach the next step. Maybe I am confused of the conditional notation and semicolon notation.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-10-22T18:39:31.297Z
Comments.Text: Where are the screenshots from?
Comments.UserDisplayName: scribe
Tags: <machine-learning><maximum-likelihood><conditional-probability>
Title: I am confused of derivation steps of MAP for linear regression
UserDisplayName: tesio
ThreadText: # I am confused of derivation steps of MAP for linear regression
  
  (Asked by: username_0 on 2023-03-08)
  
  I am taking ML course and I am confused about some derivations of math
  
  [![enter image description here](https://i.sstatic.net/ZhfSK.png)](https://i.sstatic.net/ZhfSK.png)
  [![enter image description here](https://i.sstatic.net/BBn0b.png)](https://i.sstatic.net/BBn0b.png)
  
  Could you explain the two steps I marked on the slides? For the first step, I thought $P(beta|X,y) = \frac{P(X,y|beta)P(beta)}{P(X,y)}$ but I don't know the further steps to reach the next step. Maybe I am confused of the conditional notation and semicolon notation.
  
  **Tags:** machine-learning, maximum-likelihood, conditional-probability
  > **Comments:**
  > **username_1**: Where are the screenshots from?  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2023-03-09. Score: 0 | Accepted answer
  
  **For step 1**: you're on the right track, but the derivation is easier if we leave everything conditional on $X$
  $$p(\beta| X,y) = \frac{p(y| X, \beta) p(\beta | X)}{p(y |X)}$$
  Now, the two key steps. First, $p(\beta | X) = p(\beta)$ if $X$ and $\beta$ are independent (why is this the case?). Second, $p(y |X)$ is constant in $\beta$. How does multiplying a function by a constant affect the location of its maximum?
  
  **For step 2**: simply substitute the expressions given for $p(y|X, \beta)$ and $p(\beta)$. There will be additional terms that don't depend on $\beta$. How does adding a constant to a function affect the location of its maximum?
  
  > **Comments:**
  > **username_0**: How do we get the first equality of leaving everything conditional on $X$? I don't understand what leaving everything conditional on $X$ means. Also, how do we know that $X$ and $\beta$ are independent?  
  > **username_2**: The idea is that we can apply Bayes rule to conditional distributions that are all conditioned on the same thing. In other words, if we ignored the $X$ conditioning in all terms of the above equation, we would get Bayes rule. It's also valid with the conditioning there. You can prove this by fiddling around with the definitions of conditional distributions. In terms of independence, should we expect the slope of a linear function to depend on the points that we probe it?  
  > **username_0**: I understand the conditional part. For the independence, do you mean that the slope doesn't change if the X changes? Then, the slope depends on $y$ but not $X$?  
  > **username_2**: Yes indeed.  For linear regression, the model that we are fitting is: $X  \rightarrow y \leftarrow \beta$    So $X$ and $\beta$ are marginally independent. Note that in standard linear regression,  $X$ is not even a random variable, but rather is assumed to be known exactly  
  
  ---
  
  