Answers.Body: <p>Some lines above the author says </p>
  
  <blockquote>
    <p>By the law of large numbers we can also assume that the total input to the node has a Gaussian distribution </p>
  </blockquote>
  
  <p>hence we can assume <span class="math-container">$X \sim \mathcal{N}(0,1)$</span> with the <span class="math-container">$X$</span> domain being <strong>continuous</strong> </p>
  
  <p>Then he says the input vector is assumed to be binary which changes the domain <strong>from continuous to discrete</strong> so we can discretize it assuming <span class="math-container">$-1 \le X \le 1$</span> is mapped into zero and <span class="math-container">$X&lt; -1 $</span> and <span class="math-container">$X &gt; 1$</span> are mapped to 1 </p>
  
  <p>Finally according to the <a href="https://en.wikipedia.org/wiki/68%E2%80%9395%E2%80%9399.7_rule" rel="nofollow noreferrer">68-95-99.7 Rule</a> we can compute </p>
  
  <p><span class="math-container">$$ E(X^2) = P(X=1) \cdot 1^2 + P(X=0) \cdot 0^2 = 0.32 $$</span></p>
  
  <p>Finally probably the author rounds this up to <span class="math-container">$\frac{1}{3} \simeq 0.33$</span> </p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-04-24T11:18:10.333Z
Answers.LastActivityDate: 2019-04-24T11:18:10.333Z
Answers.UserDisplayName: Nicola Bernini
Body: <p><a href="https://page.mi.fu-berlin.de/rojas/neural/chapter/K8.pdf#page=16" rel="nofollow noreferrer">Raul Rojas' Neural Networks A Systematic Introduction, section 8.2.1</a> calculates the variance of the output of a hidden neuron.</p>
  
  <p>Raul Rojas says that "for binary vectors we have <span class="math-container">$E[x_i^2] = \frac{1}{3}$</span>" where <span class="math-container">$x_i$</span> is the input value transported through each edge to a node.</p>
  
  <p>I don't quite get how he reaches this result.</p>
  
  <p>Thank you for your time :)</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-04-24T04:47:46.087Z
Comments.Text: The integral from 0 to 1 of x^2dx is 1/3, but isn't this related to a continuous random variable, instead of a binary one?
Comments.UserDisplayName: Jaume Oliver Lafont
Tags: <neural-networks><probability-distribution>
Title: Binary vector expected value
UserDisplayName: EmmanuelMess
ThreadText: # Binary vector expected value
  
  (Asked by: username_0 on 2019-04-23)
  
  [Raul Rojas' Neural Networks A Systematic Introduction, section 8.2.1](https://page.mi.fu-berlin.de/rojas/neural/chapter/K8.pdf#page=16) calculates the variance of the output of a hidden neuron.
  
  Raul Rojas says that "for binary vectors we have $E[x\_i^2] = \frac{1}{3}$" where $x\_i$ is the input value transported through each edge to a node.
  
  I don't quite get how he reaches this result.
  
  Thank you for your time :)
  
  **Tags:** neural-networks, probability-distribution
  
  > **Comments:**
  > **username_1**: The integral from 0 to 1 of x^2dx is 1/3, but isn't this related to a continuous random variable, instead of a binary one?  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2019-04-24. Score: 2 | Accepted answer
  
  Some lines above the author says
  
  > By the law of large numbers we can also assume that the total input to the node has a Gaussian distribution
  
  hence we can assume $X \sim \mathcal{N}(0,1)$ with the $X$ domain being **continuous**
  
  Then he says the input vector is assumed to be binary which changes the domain **from continuous to discrete** so we can discretize it assuming $-1 \le X \le 1$ is mapped into zero and $X< -1 $ and $X > 1$ are mapped to 1
  
  Finally according to the [68-95-99.7 Rule](https://en.wikipedia.org/wiki/68%E2%80%9395%E2%80%9399.7_rule) we can compute
  
  $$ E(X^2) = P(X=1) \cdot 1^2 + P(X=0) \cdot 0^2 = 0.32 $$
  
  Finally probably the author rounds this up to $\frac{1}{3} \simeq 0.33$
  
  
  ---
  
  