Random Initialization, Deep Learning.
for larger neural networks, let’s say of three features and maybe a very large number of hidden units, a similar argument works to show that with a neural network like this. Let me draw all the edges, if you initialize the weights to zero, then all of your hidden units are symmetric. And no matter