Operational calculus on programming spaces and generalized tensor networks
Weird I was just looking at this last night.  They implemented it: <a href="https://github.com/ZigaSajovic/dCpp" rel="nofollow">https://github.com/ZigaSajovic/dCpp</a><p>I got onto that based on this fantasy I was having.  Basically I keep thinking that the main advantage deep learning has is probably not actually the neural nets, but the fact that they are highly parallel and very general computation techniques.<p>So if it doesn't really have to be a NN, maybe there could be a way to more efficiently achieve the same level of results by using something more like evolved parallel programs. Maybe have a universal virtual sensor grounding. Then if you find a useful/general procedure/dataset you could distribute it to improve lots of diverse agent's knowledge/skills immediately.<p>But to evolve these supposed universal function-approximating parallel programs that are more efficient and exchangable than deep NNs, you would need something like backpropagation, except for some arbitrary program structures.   So when reading about backpropagation on wikipedia they mention the generalization is <a href="https://en.m.wikipedia.org/wiki/Automatic_differentiation" rel="nofollow">https://en.m.wikipedia.org/wiki/Automatic_differentiation</a><p>Anyway I don't grock all the details fully but my interpretation of all of this is that I think things like backpropagation with deep NNs are truly general purpose and when combined with grounding such as streams of virtual/real senses and training across widely varied tasks can actually produce general human and animal/like intelligence.  I think the problem is it just takes a ton of computing power, iterations and time.  If we wait 2-10 years that probably won't matter.  But if we don't want to wait we can probably do something similar and more efficient/readily distributable if we can use algorithms that aren't NNs but are general and highly parallel and can  be adjusted via AD to learn.<p>Based on this I personally believe we will see human/animal like general intelligence systems before 2020.