title: Why should weights of Neural Networks be initialized to random numbers?
body: <p>I am trying to build a neural network from scratch.
  Across all AI literature there is a consensus that weights should be initialized to random numbers in order for the network to converge faster.</p>
  
  <p><strong>But why are neural networks initial weights initialized as random numbers?</strong> </p>
  
  <p>I had read somewhere that this is done to "break the symmetry" and this makes the neural network learn faster. How does breaking the symmetry make it learn faster?</p>
  
  <p>Wouldn't initializing the weights to 0 be a better idea? That way the weights would be able to find their values (whether positive or negative) faster?</p>
  
  <p>Is there some other underlying philosophy behind randomizing the weights apart from hoping that they would be near their optimum values when initialized?</p>