Artificial Intelligence Software Is Booming, But Why Now?
Because it&#x27;s been just about 10 years since they figured out how to:<p>1) Use convolutional layers, ReLUs, and a few other tricks to ameliorate the vanishing gradient problem,<p>2) Perform continuous, high-dimensional stochastic gradient descent on graphics cards, and<p>3) Apply these things via stochastic grad-student descent to sufficiently massive datasets that even the most brute-force models and training methods (backpropagation of errors on a loss functional) can actually work.<p>In the meantime, the hardware for doing it has become commoditized and the software has consolidated and become standardized.  So now it&#x27;s A Thing in industry.<p>There are lots of &quot;smarter&quot; algorithms that almost definitely come closer to human cognition, for instance probabilistic program induction.  But <i>those</i> aren&#x27;t <i>fast</i>, and don&#x27;t always neatly separate training from prediction: you&#x27;re just not gonna be able to train those models ahead-of-time on a 10,000 image corpus inside a single week with today&#x27;s hardware and software.<p>We need to find ways to make machine learning fast even when it&#x27;s not just a bunch of matrix multiplies.  Otherwise, every time we make our algorithms more interesting, we cripple ourselves computationally.