Answers.Body: <p>Randomness is typically the best one can do with ignorance, rather than a source of strength in its own right.</p>
  
  <p>For example, the primary use of randomness in statistics is random assignment (A/B testing, randomized controlled trials, etc.). The reason to do this is to make the influence of confounders independent from the influence of the factor under investigation.</p>
  
  <p>But randomness only works for this <em>in expectation</em>. If we actually knew what the confounders were, we could do a paired assignment (or a similar scheme) that ensured the various groups were matched <em>as well as possible</em>, instead of us just not knowing ahead of time which way the bias went.</p>
  
  <hr>
  
  <p>There are some cases where pseudorandomness, rather than full randomness, will impair training AI designs. A simple example would be a case where you want to randomly initialize weights in a network where the number of parameters exceeds the periodicity of the RNG; this means that while you have as many possible networks as there are possible unique seeds, you can't actually visit the entire weight space that you wanted to sample over.</p>
  
  <p>I don't think any of those cases are limiting factors, however. Having truly random stochastic gradient descent instead of pseudorandom stochastic gradient descent doesn't seem like it would make a serious difference in the trajectory of AI designs.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-03-12T20:48:14.940Z
Answers.Text: @NietzscheanAI I think that's a really good question. I suggest you post that as a proper question in ai.stackexchange
Answers.UserDisplayName: Hugo Nava Kopp
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-08-04T19:41:31.323Z
Answers.Text: Is the number of parameters an issue, given that a modern RNG like Mersenne Twister has a period of (2^19968) - 1?
Answers.UserDisplayName: NietzscheanAI
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-08-05T13:39:36.833Z
Answers.Text: @user217281728 I'm starting to step outside of my area of expertise, but I'd guess it wouldn't for something like the Mersenne Twister. (That would require a truly huge AI model to exhaust!)
Answers.UserDisplayName: Matthew Gray
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-08-04T18:56:47.850Z
Answers.LastActivityDate: 2016-08-04T18:56:47.850Z
Answers.UserDisplayName: Matthew Gray
Body: <p>How important is true (non-<a href="https://en.wikipedia.org/wiki/Pseudorandomness" rel="noreferrer" title="pseudo">pseudo</a>) randomness in Artificial Intelligence designs? Is there any chance that pseudo-randomness could be a barrier to more successful designs?</p>
  
Tags: <ai-design>
Title: How important is true randomness in AI designs?
UserDisplayName: dynrepsys
ThreadText: # How important is true randomness in AI designs?
  
  (Asked by: username_0 on 2016-08-04)
  
  How important is true (non-[pseudo](https://en.wikipedia.org/wiki/Pseudorandomness "pseudo")) randomness in Artificial Intelligence designs? Is there any chance that pseudo-randomness could be a barrier to more successful designs?
  
  **Tags:** ai-design
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2016-08-04. Score: 5
  
  Randomness is typically the best one can do with ignorance, rather than a source of strength in its own right.
  
  For example, the primary use of randomness in statistics is random assignment (A/B testing, randomized controlled trials, etc.). The reason to do this is to make the influence of confounders independent from the influence of the factor under investigation.
  
  But randomness only works for this *in expectation*. If we actually knew what the confounders were, we could do a paired assignment (or a similar scheme) that ensured the various groups were matched *as well as possible*, instead of us just not knowing ahead of time which way the bias went.
  
  ---
  
  There are some cases where pseudorandomness, rather than full randomness, will impair training AI designs. A simple example would be a case where you want to randomly initialize weights in a network where the number of parameters exceeds the periodicity of the RNG; this means that while you have as many possible networks as there are possible unique seeds, you can't actually visit the entire weight space that you wanted to sample over.
  
  I don't think any of those cases are limiting factors, however. Having truly random stochastic gradient descent instead of pseudorandom stochastic gradient descent doesn't seem like it would make a serious difference in the trajectory of AI designs.
  
  > **Comments:**
  > **username_3**: Is the number of parameters an issue, given that a modern RNG like Mersenne Twister has a period of (2^19968) - 1?  
  > **username_1**: @user217281728 I'm starting to step outside of my area of expertise, but I'd guess it wouldn't for something like the Mersenne Twister. (That would require a truly huge AI model to exhaust!)  
  > **username_2**: @NietzscheanAI I think that's a really good question. I suggest you post that as a proper question in ai.stackexchange  
  
  ---
  
  