Answers.Body: <p>You're taking an expectation of a function <span class="math-container">$f$</span>, which depends on 2 variables (including <span class="math-container">$z$</span>), with respect to <span class="math-container">$z$</span>.</p>
  <p>The general formula for <a href="https://en.wikipedia.org/wiki/Expected_value" rel="nofollow noreferrer">an expectation as an integral</a> is <span class="math-container">$E_z[f(z)] = \int p(z) f(z) dz$</span>, where <span class="math-container">$p$</span> is the density.</p>
  <p>See <a href="https://en.wikipedia.org/wiki/Law_of_the_unconscious_statistician" rel="nofollow noreferrer">the law of unconscious statistician</a>.</p>
  <p>So, it's not <span class="math-container">$p(x, y) = p(x | y) p(y)$</span>. That's something different.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-12-02T12:55:38.530Z
Answers.LastActivityDate: 2024-12-02T12:55:38.530Z
Answers.UserDisplayName: nbro
Answers.Body: <p>Your green box derivation is only valid for computing marginal distribution <span class="math-container">$f(x)$</span> from the joint probability distribution <span class="math-container">$f(x,z)$</span> via chain rule, <em>assuming</em> <span class="math-container">$f(x,z)$</span> is a joint distribution first. However, here for test time drop out to rid of random result from random mask <span class="math-container">$z$</span>, the neural network mapping <span class="math-container">$f(x,z)$</span> is <em>not</em> a joint distribution but some deterministic universal approximating function.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-12-02T19:19:27.963Z
Answers.LastActivityDate: 2024-12-02T19:30:54.060Z
Answers.LastEditDate: 2024-12-02T19:30:54.060Z
Answers.UserDisplayName: cinch
Body: <p>This following slide is from Deep Learning Lecture by Justin Johnson that is accessible on Youtube.</p>
  <p>This is part is about how to handle dropout during test time.
  I derived the equation just like the green box, but it is different from the original text. What I have done wrong?</p>
  <p><a href="https://i.sstatic.net/E4hO241Z.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/E4hO241Z.jpg" alt="enter image description here" /></a></p>
  
Tags: <dropout>
Title: dropout test phase probability calculation
UserDisplayName: lcdreamer621
ThreadText: # dropout test phase probability calculation
  
  (Asked by: username_0 on 2024-12-02)
  
  This following slide is from Deep Learning Lecture by Justin Johnson that is accessible on Youtube.
  
  This is part is about how to handle dropout during test time.
  I derived the equation just like the green box, but it is different from the original text. What I have done wrong?
  
  [![enter image description here](https://i.sstatic.net/E4hO241Z.jpg)](https://i.sstatic.net/E4hO241Z.jpg)
  
  **Tags:** dropout
  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2024-12-02. Score: 1
  
  You're taking an expectation of a function $f$, which depends on 2 variables (including $z$), with respect to $z$.
  
  The general formula for [an expectation as an integral](https://en.wikipedia.org/wiki/Expected_value) is $E\_z[f(z)] = \int p(z) f(z) dz$, where $p$ is the density.
  
  See [the law of unconscious statistician](https://en.wikipedia.org/wiki/Law_of_the_unconscious_statistician).
  
  So, it's not $p(x, y) = p(x | y) p(y)$. That's something different.
  
  
  ---
  
  ### Answer by username_2 on 2024-12-02. Score: 1
  
  Your green box derivation is only valid for computing marginal distribution $f(x)$ from the joint probability distribution $f(x,z)$ via chain rule, *assuming* $f(x,z)$ is a joint distribution first. However, here for test time drop out to rid of random result from random mask $z$, the neural network mapping $f(x,z)$ is *not* a joint distribution but some deterministic universal approximating function.
  
  
  ---
  
  