Answers.Body: <p>When you use a particular seed, it actually ceases to become a random initialization and is instead fixed. I believe the only reason to actually do this would be for reliable reproduction in research and not as a method of training production models.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-11-16T07:13:47.583Z
Answers.Text: But why doesn't it work? If the "random" initialization is important, then we would expect different convergence results for different initializations - for supervised learning this is rarely the case.
Answers.UserDisplayName: user3180
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-11-16T08:53:02.530Z
Answers.Text: different random initializations will give different convergence results (slightly different). If you read research papers, they will often cite a $\pm$ after the results which are the different values on different random initializations. They use the seed so that they can reproduce the EXACT same results again. Different random initializations without a seed will give slightly different results. If you use a fixed seed and flip a coin 100 times, you will get the exact same pattern of H and T every time with that seed. While a different random seed would give a random pattern
Answers.UserDisplayName: Joff
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-11-23T11:38:38.720Z
Answers.Text: Why would we ever care about having slightly different convergence results?
Answers.UserDisplayName: user3180
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-11-23T11:39:12.157Z
Answers.Text: You just gave me a good reasoning for why we can set and forget seed=0 in production as well. +- .001% is not going to make a difference (plus, you have to maximize over the seed to get the best result, not just randomly pick one)
Answers.UserDisplayName: user3180
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-11-14T17:21:42.250Z
Answers.LastActivityDate: 2020-11-14T17:21:42.250Z
Answers.UserDisplayName: Joff
Body: <p>For example, if we set the random seed to be 0, will we run into any problems? For example, maybe for seed 0, we can only reach a certain training error, but other seeds will converge to a much lower error</p>
  <p>I'm specifically concerned about supervised learning on point cloud data, but curious about whether it matters in general whenever you use a neural network.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-11-14T14:04:46.630Z
Comments.Text: Do you know what the seed is used for? (I did not downvote, by the way).
Comments.UserDisplayName: nbro
Tags: <neural-networks><training><weights-initialization>
Title: Are there any downsides of using a fixed seed for a neural network's weight initialization?
UserDisplayName: user3180
ThreadText: # Are there any downsides of using a fixed seed for a neural network's weight initialization?
  
  (Asked by: username_0 on 2020-11-14)
  
  For example, if we set the random seed to be 0, will we run into any problems? For example, maybe for seed 0, we can only reach a certain training error, but other seeds will converge to a much lower error
  
  I'm specifically concerned about supervised learning on point cloud data, but curious about whether it matters in general whenever you use a neural network.
  
  **Tags:** neural-networks, training, weights-initialization
  > **Comments:**
  > **username_1**: Do you know what the seed is used for? (I did not downvote, by the way).  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2020-11-14. Score: 1
  
  When you use a particular seed, it actually ceases to become a random initialization and is instead fixed. I believe the only reason to actually do this would be for reliable reproduction in research and not as a method of training production models.
  
  > **Comments:**
  > **username_0**: But why doesn't it work? If the "random" initialization is important, then we would expect different convergence results for different initializations - for supervised learning this is rarely the case.  
  > **username_2**: different random initializations will give different convergence results (slightly different). If you read research papers, they will often cite a $\pm$ after the results which are the different values on different random initializations. They use the seed so that they can reproduce the EXACT same results again. Different random initializations without a seed will give slightly different results. If you use a fixed seed and flip a coin 100 times, you will get the exact same pattern of H and T every time with that seed. While a different random seed would give a random pattern  
  > **username_0**: Why would we ever care about having slightly different convergence results?  
  > **username_0**: You just gave me a good reasoning for why we can set and forget seed=0 in production as well. +- .001% is not going to make a difference (plus, you have to maximize over the seed to get the best result, not just randomly pick one)  
  
  ---
  
  