Neural Networks, Manifolds, and Topology
This is a very interesting perspective on neural networks, is it novel? I've never seen a geometric interpretation of NNs.<p>In particular this:<p>> (Apparently determining if knots are trivial is NP. This doesn’t bode well for neural networks.)<p>Is there any theoretical research on obstructions to NN learning? Not that it would change much the practice (for instance, MLE learning of gaussian mixtures is NP-hard, but everybody does it anyway), but it could shed some light on the geometry of the "hard" instances.<p>EDIT: for example, to get a sense of how ill-conditioned is deep NN learning, a recent paper [1] shows that if we feed to an object classifying NN an image with crafted but unnoticeable perturbations the predictions change completely.<p>[1] <a href="http://arxiv.org/abs/1312.6199" rel="nofollow">http://arxiv.org/abs/1312.6199</a>