<p>You're taking an expectation of a function <span class="math-container">$f$</span>, which depends on 2 variables (including <span class="math-container">$z$</span>), with respect to <span class="math-container">$z$</span>.</p>
<p>The general formula for <a href="https://en.wikipedia.org/wiki/Expected_value" rel="nofollow noreferrer">an expectation as an integral</a> is <span class="math-container">$E_z[f(z)] = \int p(z) f(z) dz$</span>, where <span class="math-container">$p$</span> is the density.</p>
<p>See <a href="https://en.wikipedia.org/wiki/Law_of_the_unconscious_statistician" rel="nofollow noreferrer">the law of unconscious statistician</a>.</p>
<p>So, it's not <span class="math-container">$p(x, y) = p(x | y) p(y)$</span>. That's something different.</p>

CC BY-SA 4.0
2024-12-02T12:55:38.530Z
2024-12-02T12:55:38.530Z
nbro
<p>Your green box derivation is only valid for computing marginal distribution <span class="math-container">$f(x)$</span> from the joint probability distribution <span class="math-container">$f(x,z)$</span> via chain rule, <em>assuming</em> <span class="math-container">$f(x,z)$</span> is a joint distribution first. However, here for test time drop out to rid of random result from random mask <span class="math-container">$z$</span>, the neural network mapping <span class="math-container">$f(x,z)$</span> is <em>not</em> a joint distribution but some deterministic universal approximating function.</p>

CC BY-SA 4.0
2024-12-02T19:19:27.963Z
2024-12-02T19:30:54.060Z
2024-12-02T19:30:54.060Z
cinch
<p>This following slide is from Deep Learning Lecture by Justin Johnson that is accessible on Youtube.</p>
<p>This is part is about how to handle dropout during test time.
I derived the equation just like the green box, but it is different from the original text. What I have done wrong?</p>
<p><a href="https://i.sstatic.net/E4hO241Z.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/E4hO241Z.jpg" alt="enter image description here" /></a></p>

<dropout>
dropout test phase probability calculation
lcdreamer621
# dropout test phase probability calculation

(Asked by: username_0 on 2024-12-02)

This following slide is from Deep Learning Lecture by Justin Johnson that is accessible on Youtube.

This is part is about how to handle dropout during test time.
I derived the equation just like the green box, but it is different from the original text. What I have done wrong?

[![enter image description here](https://i.sstatic.net/E4hO241Z.jpg)](https://i.sstatic.net/E4hO241Z.jpg)

**Tags:** dropout


---

## 2 answers

### Answer by username_1 on 2024-12-02. Score: 1

You're taking an expectation of a function $f$, which depends on 2 variables (including $z$), with respect to $z$.

The general formula for [an expectation as an integral](https://en.wikipedia.org/wiki/Expected_value) is $E\_z[f(z)] = \int p(z) f(z) dz$, where $p$ is the density.

See [the law of unconscious statistician](https://en.wikipedia.org/wiki/Law_of_the_unconscious_statistician).

So, it's not $p(x, y) = p(x | y) p(y)$. That's something different.


---

### Answer by username_2 on 2024-12-02. Score: 1

Your green box derivation is only valid for computing marginal distribution $f(x)$ from the joint probability distribution $f(x,z)$ via chain rule, *assuming* $f(x,z)$ is a joint distribution first. However, here for test time drop out to rid of random result from random mask $z$, the neural network mapping $f(x,z)$ is *not* a joint distribution but some deterministic universal approximating function.


---

