I can chime in for the theoretical computer scientists. Deep learning (in its simplest form) corresponds to the class of circuits whose gates are linear threshold functions. Our primary goal with such functions is not to show what problems can be solved by small circuits using linear threshold gates, but what problems <i>cannot</i> be solved with such circuits.<p>Until last week [1], it was an open problem whether every function with n inputs that is computable in <i>nondeterministic</i> time 2^O(n) could also be computed with a two-layer circuit using only O(n) gates (that is, a deep net with just one hidden layer with a linear number of gates). This is an embarrassing state of knowledge.<p>Now we know the following slightly less embarrassing thing: there is an explicit function, computable in linear time, that needs at least n^3&#x2F;2 gates in a two-layer deep net (modulo some log(n) factors), and another function that needs n^3&#x2F;2 gates in a three-layer deep net (with the additional restriction that the output gate is a majority vote of the previous layer).<p>This is still a long way away from truly understanding the representational power of deep nets, but it&#x27;s the first general progress that has been made since the early 90&#x27;s.<p>[1]: <a href="http:&#x2F;&#x2F;eccc.hpi-web.de&#x2F;report&#x2F;2015&#x2F;188&#x2F;" rel="nofollow">http:&#x2F;&#x2F;eccc.hpi-web.de&#x2F;report&#x2F;2015&#x2F;188&#x2F;</a>