It really kinda is.<p>In the Ilya Sutskever <i>Talking Machines</i> podcast[1] he describes exactly how it is &quot;magic&quot;. He talks about how there is no theoretical basis to think that a deep neural network <i>should</i> converge, and prior to around 2006 the accepted wisdom was that networks deep enough to outperform other methods of machine learning were useless because they couldn&#x27;t be trained.<p>And then they discovered how to initialize the random starting values in such a way that they <i>do</i> converge - for reasons no one really understands.<p>I&#x27;ve heard a lot of people claim there is no magic, but I tend to think Sutskever[2] knows a little bit more than most people about deep neural networks.<p>[1] <a href="http:&#x2F;&#x2F;www.thetalkingmachines.com&#x2F;blog&#x2F;2015&#x2F;1&#x2F;15&#x2F;machine-learning-and-magical-thinking" rel="nofollow">http:&#x2F;&#x2F;www.thetalkingmachines.com&#x2F;blog&#x2F;2015&#x2F;1&#x2F;15&#x2F;machine-lea...</a><p>[2] <a href="http:&#x2F;&#x2F;www.cs.toronto.edu&#x2F;~ilya&#x2F;" rel="nofollow">http:&#x2F;&#x2F;www.cs.toronto.edu&#x2F;~ilya&#x2F;</a>