Neural Networks, Manifolds, and Topology
This is a very interesting perspective on neural networks, is it novel? I've never seen a geometric interpretation of NNs.In particular this:> (Apparently determining if knots are trivial is NP. This doesn’t bode well for neural networks.)Is there any theoretical research on obstructions to NN learning? Not that it would change much the practice (for instance, MLE learning of gaussian mixtures is NP-hard, but everybody does it anyway), but it could shed some light on the geometry of the "hard" instances.EDIT: for example, to get a sense of how ill-conditioned is deep NN learning, a recent paper [1] shows that if we feed to an object classifying NN an image with crafted but unnoticeable perturbations the predictions change completely.[1] http://arxiv.org/abs/1312.6199