<p>Some lines above the author says </p>

<blockquote>
  <p>By the law of large numbers we can also assume that the total input to the node has a Gaussian distribution </p>
</blockquote>

<p>hence we can assume <span class="math-container">$X \sim \mathcal{N}(0,1)$</span> with the <span class="math-container">$X$</span> domain being <strong>continuous</strong> </p>

<p>Then he says the input vector is assumed to be binary which changes the domain <strong>from continuous to discrete</strong> so we can discretize it assuming <span class="math-container">$-1 \le X \le 1$</span> is mapped into zero and <span class="math-container">$X&lt; -1 $</span> and <span class="math-container">$X &gt; 1$</span> are mapped to 1 </p>

<p>Finally according to the <a href="https://en.wikipedia.org/wiki/68%E2%80%9395%E2%80%9399.7_rule" rel="nofollow noreferrer">68-95-99.7 Rule</a> we can compute </p>

<p><span class="math-container">$$ E(X^2) = P(X=1) \cdot 1^2 + P(X=0) \cdot 0^2 = 0.32 $$</span></p>

<p>Finally probably the author rounds this up to <span class="math-container">$\frac{1}{3} \simeq 0.33$</span> </p>

CC BY-SA 4.0
2019-04-24T11:18:10.333Z
2019-04-24T11:18:10.333Z
Nicola Bernini
<p><a href="https://page.mi.fu-berlin.de/rojas/neural/chapter/K8.pdf#page=16" rel="nofollow noreferrer">Raul Rojas' Neural Networks A Systematic Introduction, section 8.2.1</a> calculates the variance of the output of a hidden neuron.</p>

<p>Raul Rojas says that "for binary vectors we have <span class="math-container">$E[x_i^2] = \frac{1}{3}$</span>" where <span class="math-container">$x_i$</span> is the input value transported through each edge to a node.</p>

<p>I don't quite get how he reaches this result.</p>

<p>Thank you for your time :)</p>

CC BY-SA 4.0
2019-04-24T04:47:46.087Z
The integral from 0 to 1 of x^2dx is 1/3, but isn't this related to a continuous random variable, instead of a binary one?
Jaume Oliver Lafont
<neural-networks><probability-distribution>
Binary vector expected value
EmmanuelMess
# Binary vector expected value

(Asked by: username_0 on 2019-04-23)

[Raul Rojas' Neural Networks A Systematic Introduction, section 8.2.1](https://page.mi.fu-berlin.de/rojas/neural/chapter/K8.pdf#page=16) calculates the variance of the output of a hidden neuron.

Raul Rojas says that "for binary vectors we have $E[x\_i^2] = \frac{1}{3}$" where $x\_i$ is the input value transported through each edge to a node.

I don't quite get how he reaches this result.

Thank you for your time :)

**Tags:** neural-networks, probability-distribution

> **Comments:**
> **username_1**: The integral from 0 to 1 of x^2dx is 1/3, but isn't this related to a continuous random variable, instead of a binary one?  

---

## 1 answers

### Answer by username_2 on 2019-04-24. Score: 2 | Accepted answer

Some lines above the author says

> By the law of large numbers we can also assume that the total input to the node has a Gaussian distribution

hence we can assume $X \sim \mathcal{N}(0,1)$ with the $X$ domain being **continuous**

Then he says the input vector is assumed to be binary which changes the domain **from continuous to discrete** so we can discretize it assuming $-1 \le X \le 1$ is mapped into zero and $X< -1 $ and $X > 1$ are mapped to 1

Finally according to the [68-95-99.7 Rule](https://en.wikipedia.org/wiki/68%E2%80%9395%E2%80%9399.7_rule) we can compute

$$ E(X^2) = P(X=1) \cdot 1^2 + P(X=0) \cdot 0^2 = 0.32 $$

Finally probably the author rounds this up to $\frac{1}{3} \simeq 0.33$


---

