&gt; Seeing different ways to look at something is always refreshing ...  how do YOU think about NNs?<p>A year or two ago, when I was really doing the wandering researcher thing, I made a point of asking lots of people at different groups how they saw deep learning. I think these three narratives capture most of what I heard, although a people blended them in different ways.<p>Certain views are more common in particular groups. For example, people at the Montreal group seem more inclined to think about things from the representations narrative than other groups. (Although, I think I have just about everyone else beat in my extreme version of seeing everything from the manifold perspective. :P )<p>I feel like there&#x27;s some pretty interesting quasi-sociological work to be done on deep learning. Because the community has grown so fast, and there&#x27;s a lot of variety in how people think.<p>&gt;&gt; ... chain of composed functions is optimized..
&gt; This is certainly true for the majority of models in use today, but I wonder if this will remain true for future architectures.<p>I suspect there&#x27;s something very fundamental about chains of composed functions. I can&#x27;t formalize my feeling into a strong argument though.<p>&gt; In the case of a network that is supposed to update it&#x27;s own weights during normal operation (i.e. topic discovery), the functional narrative is less clear to me.<p>I don&#x27;t think matters too much. The functional graph becomes more complicated, but it&#x27;s just a matter of the output of one function going to multiple places.