<i>This is still a long way away from truly understanding the representational power of deep nets, but it&#x27;s the first general progress that has been made since the early 90&#x27;s.</i><p>Exactly. Someone should tell Yann LeCun this --see <i>e.g.</i> Section 3.2 in [1], or pretty much every time he brings up circuit complexity theory to automagically imply that &quot;most functions representable compactly with a deep architecture would require a very large number of components if represented with a shallow one.&quot; [<i>ibid</i>, p.14]<p>[1] <a href="http:&#x2F;&#x2F;yann.lecun.com&#x2F;exdb&#x2F;publis&#x2F;pdf&#x2F;bengio-lecun-07.pdf" rel="nofollow">http:&#x2F;&#x2F;yann.lecun.com&#x2F;exdb&#x2F;publis&#x2F;pdf&#x2F;bengio-lecun-07.pdf</a>