Answers.Body: <p>In expectation step, firstly we calculate the posterior of latent variable <span class="math-container">$Z$</span> and then the <strong><span class="math-container">$Q(θ | θ^{(t)})$</span> is defined as the expected value of the log likelihood of <span class="math-container">$θ$</span>, with respect to the current conditional contribution of <span class="math-container">$Z$</span> given <span class="math-container">$X$</span> and the current estimates of <span class="math-container">$θ^{(t)}$</span></strong>. In maximization step, we update <span class="math-container">$θ$</span> using the argmax on <span class="math-container">$Q$</span>, with respect to <span class="math-container">$θ$</span>.</p>
  <p><span class="math-container">$$Q(θ | θ^{(t)}) = E_{Z|X,θ^{(t)}}[logL(θ;Χ,Z)]$$</span></p>
  <p>To be more intuitive, think of k-means as a special case of EM, where in expectation step the <span class="math-container">$Z$</span> variables are defined, that is the latent variables indicating the membership in a cluster, and calculated in a hard assignment way. In maximization step the <span class="math-container">$μ$</span>s of the clusters are updated. If you want to see the corresponding relation for <span class="math-container">$Q$</span> in k-means, I suggest you read the chapter 9.3.2 in C.Bishop's book: Pattern Recognition and Machine Learning.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-10-07T03:10:46.553Z
Answers.Text: I understood the intuition behind the E step in k means, but is it possible to extend the same intuition to the general EM algorithm?How do we visualise the posterior as an expectation in the derivation of the general EM algorithm? Or equivalently, can we show that the expectation term in the formula given by you for the E step can be expressed as the posterior?
Answers.UserDisplayName: Manish Kausik Hari Baskar
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-10-07T08:16:37.023Z
Answers.Text: @Manish Kausik Hari Baskar. The Q function uses the posterior calculated in E step as the sum of posteriors * $ln(X,Z|θ)$ over Z. Also, I fixed my post.
Answers.UserDisplayName: ddaedalus
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-10-06T22:21:58.907Z
Answers.LastActivityDate: 2020-10-07T08:12:05.223Z
Answers.LastEditDate: 2020-10-07T08:12:05.223Z
Answers.UserDisplayName: ddaedalus
Body: <p>The E step on the EM algorithm asks us to set the value of the variational lower bound to be equal to the posterior probability of the latent variable, given the data points and parameters. Clearly we are not taking any expectations here, then why is it called the Expectation step? Am I missing something here?</p>
  
Tags: <bayesian-probability><expectation-maximization><bayesian-neural-networks>
Title: Why is the E step in expectation maximisation algorithm called so?
UserDisplayName: Manish Kausik Hari Baskar
ThreadText: # Why is the E step in expectation maximisation algorithm called so?
  
  (Asked by: username_0 on 2020-10-06)
  
  The E step on the EM algorithm asks us to set the value of the variational lower bound to be equal to the posterior probability of the latent variable, given the data points and parameters. Clearly we are not taking any expectations here, then why is it called the Expectation step? Am I missing something here?
  
  **Tags:** bayesian-probability, expectation-maximization, bayesian-neural-networks
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-10-06. Score: 1 | Accepted answer
  
  In expectation step, firstly we calculate the posterior of latent variable $Z$ and then the **$Q(θ | θ^{(t)})$ is defined as the expected value of the log likelihood of $θ$, with respect to the current conditional contribution of $Z$ given $X$ and the current estimates of $θ^{(t)}$**. In maximization step, we update $θ$ using the argmax on $Q$, with respect to $θ$.
  
  $$Q(θ | θ^{(t)}) = E\_{Z|X,θ^{(t)}}[logL(θ;Χ,Z)]$$
  
  To be more intuitive, think of k-means as a special case of EM, where in expectation step the $Z$ variables are defined, that is the latent variables indicating the membership in a cluster, and calculated in a hard assignment way. In maximization step the $μ$s of the clusters are updated. If you want to see the corresponding relation for $Q$ in k-means, I suggest you read the chapter 9.3.2 in C.Bishop's book: Pattern Recognition and Machine Learning.
  
  > **Comments:**
  > **username_0**: I understood the intuition behind the E step in k means, but is it possible to extend the same intuition to the general EM algorithm?How do we visualise the posterior as an expectation in the derivation of the general EM algorithm? Or equivalently, can we show that the expectation term in the formula given by you for the E step can be expressed as the posterior?  
  > **username_1**: @Manish Kausik Hari Baskar. The Q function uses the posterior calculated in E step as the sum of posteriors * $ln(X,Z|θ)$ over Z. Also, I fixed my post.  
  
  ---
  
  