Answers.Body: <p>This is not so difficult (just a bit verbose if you do all steps). Just replace <span class="math-container">$\mathcal{N}\left(x_{n} \mid \mu, \sigma^{2}\right)$</span> with <span class="math-container">$\frac{1}{\left(2 \pi \sigma^{2}\right)^{1 / 2}} \exp \left\{-\frac{1}{2 \sigma^{2}}(x-\mu)^{2}\right\}$</span> and apply the properties of logs (specifically, log of multiplication, log of exponential and log of a fraction). Try first before looking at my solution (otherwise you don't learn anything)!</p>
  <blockquote class="spoiler">
  <p> <span class="math-container">\begin{align} p\left(\mathbf{x} \mid \mu, \sigma^{2}\right) &amp;= \prod_{n=1}^{N} \mathcal{N}\left(x_{n} \mid \mu, \sigma^{2}\right)\\ &amp;= \prod_{n=1}^{N}\frac{1}{\left(2 \pi \sigma^{2}\right)^{1 / 2}} \exp \left\{-\frac{1}{2 \sigma^{2}}(x-\mu)^{2}\right\}  \iff \\ \log p\left(\mathbf{x} \mid \mu, \sigma^{2}\right) &amp;= \log \left( \prod_{n=1}^{N}\frac{1}{\left(2 \pi \sigma^{2}\right)^{1 / 2}} \exp \left\{-\frac{1}{2 \sigma^{2}}(x-\mu)^{2}\right\} \right) \\ &amp;= \sum_{n=1}^{N} \log \left(\frac{1}{\left(2 \pi \sigma^{2}\right)^{1 / 2}} \exp \left\{-\frac{1}{2 \sigma^{2}}(x-\mu)^{2}\right\} \right) \\ &amp;= \sum_{n=1}^{N} \left( \log \left(\frac{1}{\left(2 \pi \sigma^{2}\right)^{1 / 2}} \right) + \log \left( \exp \left\{-\frac{1}{2 \sigma^{2}}(x-\mu)^{2}\right\} \right) \right) \\ &amp;= \sum_{n=1}^{N} \left( - \frac{1}{2}\log \left(2 \pi \sigma^{2}\right)  -\frac{1}{2 \sigma^{2}}(x-\mu)^{2}  \right) \\ &amp;= - \frac{1}{2} \sum_{n=1}^{N} \log \left(2 \pi \sigma^{2}\right) - \frac{1}{2 \sigma^{2}} \sum_{n=1}^{N}(x-\mu)^{2} \\ &amp;= - \frac{1}{2} \sum_{n=1}^{N} \left( \log 2 \pi + \log \sigma^{2}\right) - \frac{1}{2 \sigma^{2}} \sum_{n=1}^{N}(x-\mu)^{2} \\ &amp;= - \frac{1}{2} \sum_{n=1}^{N} \log 2 \pi  - \frac{1}{2} \sum_{n=1}^{N} \log \sigma^{2}- \frac{1}{2 \sigma^{2}} \sum_{n=1}^{N}(x-\mu)^{2} \end{align}</span></p>
  </blockquote>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-06-14T20:01:00.937Z
Answers.LastActivityDate: 2022-06-14T22:55:59.060Z
Answers.LastEditDate: 2022-06-14T22:55:59.060Z
Answers.UserDisplayName: nbro
Body: <p>I am now reading the Bishop Machine Learning Book and going through every single equation.</p>
  <p>We know that in the case of a single real-valued variable <span class="math-container">$x$</span>, the Gaussian distribution is defined by
  <span class="math-container">$$\mathcal{N}\left(x \mid \mu, \sigma^{2}\right)=\frac{1}{\left(2 \pi \sigma^{2}\right)^{1 / 2}} \exp \left\{-\frac{1}{2 \sigma^{2}}(x-\mu)^{2}\right\}$$</span></p>
  <p>Since the dataset <span class="math-container">$\mathbf{X}$</span> is i.i.d., we can therefore write the probability of the dataset, given <span class="math-container">$\mu \text { and } \sigma^{2}$</span>, in the form</p>
  <p><span class="math-container">$$p\left(\mathbf{x} \mid \mu, \sigma^{2}\right)=\prod_{n=1}^{N} \mathcal{N}\left(x_{n} \mid \mu, \sigma^{2}\right)$$</span></p>
  <p>We will resolve the numerical underflow by computing the sum of the log probabilities, therefore from the above two equations we have the following log-likelihood function,
  <span class="math-container">$$\ln p\left(\mathbf{x} \mid \mu, \sigma^{2}\right)=-\frac{1}{2 \sigma^{2}} \sum_{n=1}^{N}\left(x_{n}-\mu\right)^{2}-\frac{N}{2} \ln \sigma^{2}-\frac{N}{2} \ln (2 \pi)$$</span></p>
  <p>I am not quite sure how did they derive this in the book. Thanks a lot and have a good day.</p>
  <p>You can refer to page 27 equation 1.54 for the detail.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-06-14T20:14:07.970Z
Comments.Text: [Please, next time, provide a more descriptive title!](https://ai.meta.stackexchange.com/q/1654/2444).
Comments.UserDisplayName: nbro
Tags: <machine-learning><math><normal-distribution>
Title: How does Bishop derive $\ln p\left(\mathbf{x} \mid \mu, \sigma^{2}\right)$, when $p$ is a Gaussian?
UserDisplayName: Minty Fresh
ThreadText: # How does Bishop derive $\ln p\left(\mathbf{x} \mid \mu, \sigma^{2}\right)$, when $p$ is a Gaussian?
  
  (Asked by: username_0 on 2022-06-14)
  
  I am now reading the Bishop Machine Learning Book and going through every single equation.
  
  We know that in the case of a single real-valued variable $x$, the Gaussian distribution is defined by
  $$\mathcal{N}\left(x \mid \mu, \sigma^{2}\right)=\frac{1}{\left(2 \pi \sigma^{2}\right)^{1 / 2}} \exp \left\{-\frac{1}{2 \sigma^{2}}(x-\mu)^{2}\right\}$$
  
  Since the dataset $\mathbf{X}$ is i.i.d., we can therefore write the probability of the dataset, given $\mu \text { and } \sigma^{2}$, in the form
  
  $$p\left(\mathbf{x} \mid \mu, \sigma^{2}\right)=\prod\_{n=1}^{N} \mathcal{N}\left(x\_{n} \mid \mu, \sigma^{2}\right)$$
  
  We will resolve the numerical underflow by computing the sum of the log probabilities, therefore from the above two equations we have the following log-likelihood function,
  $$\ln p\left(\mathbf{x} \mid \mu, \sigma^{2}\right)=-\frac{1}{2 \sigma^{2}} \sum\_{n=1}^{N}\left(x\_{n}-\mu\right)^{2}-\frac{N}{2} \ln \sigma^{2}-\frac{N}{2} \ln (2 \pi)$$
  
  I am not quite sure how did they derive this in the book. Thanks a lot and have a good day.
  
  You can refer to page 27 equation 1.54 for the detail.
  
  **Tags:** machine-learning, math, normal-distribution
  > **Comments:**
  > **username_1**: [Please, next time, provide a more descriptive title!](https://ai.meta.stackexchange.com/q/1654/2444).  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2022-06-14. Score: 2 | Accepted answer
  
  This is not so difficult (just a bit verbose if you do all steps). Just replace $\mathcal{N}\left(x\_{n} \mid \mu, \sigma^{2}\right)$ with $\frac{1}{\left(2 \pi \sigma^{2}\right)^{1 / 2}} \exp \left\{-\frac{1}{2 \sigma^{2}}(x-\mu)^{2}\right\}$ and apply the properties of logs (specifically, log of multiplication, log of exponential and log of a fraction). Try first before looking at my solution (otherwise you don't learn anything)!
  
  > \begin{align} p\left(\mathbf{x} \mid \mu, \sigma^{2}\right) &= \prod\_{n=1}^{N} \mathcal{N}\left(x\_{n} \mid \mu, \sigma^{2}\right)\\ &= \prod\_{n=1}^{N}\frac{1}{\left(2 \pi \sigma^{2}\right)^{1 / 2}} \exp \left\{-\frac{1}{2 \sigma^{2}}(x-\mu)^{2}\right\} \iff \\ \log p\left(\mathbf{x} \mid \mu, \sigma^{2}\right) &= \log \left( \prod\_{n=1}^{N}\frac{1}{\left(2 \pi \sigma^{2}\right)^{1 / 2}} \exp \left\{-\frac{1}{2 \sigma^{2}}(x-\mu)^{2}\right\} \right) \\ &= \sum\_{n=1}^{N} \log \left(\frac{1}{\left(2 \pi \sigma^{2}\right)^{1 / 2}} \exp \left\{-\frac{1}{2 \sigma^{2}}(x-\mu)^{2}\right\} \right) \\ &= \sum\_{n=1}^{N} \left( \log \left(\frac{1}{\left(2 \pi \sigma^{2}\right)^{1 / 2}} \right) + \log \left( \exp \left\{-\frac{1}{2 \sigma^{2}}(x-\mu)^{2}\right\} \right) \right) \\ &= \sum\_{n=1}^{N} \left( - \frac{1}{2}\log \left(2 \pi \sigma^{2}\right) -\frac{1}{2 \sigma^{2}}(x-\mu)^{2} \right) \\ &= - \frac{1}{2} \sum\_{n=1}^{N} \log \left(2 \pi \sigma^{2}\right) - \frac{1}{2 \sigma^{2}} \sum\_{n=1}^{N}(x-\mu)^{2} \\ &= - \frac{1}{2} \sum\_{n=1}^{N} \left( \log 2 \pi + \log \sigma^{2}\right) - \frac{1}{2 \sigma^{2}} \sum\_{n=1}^{N}(x-\mu)^{2} \\ &= - \frac{1}{2} \sum\_{n=1}^{N} \log 2 \pi - \frac{1}{2} \sum\_{n=1}^{N} \log \sigma^{2}- \frac{1}{2 \sigma^{2}} \sum\_{n=1}^{N}(x-\mu)^{2} \end{align}
  
  
  ---
  
  