The issue is that NNs are continuous, by design. The more linear and continuous they are, the easier they are to train and generalize well.<p>But what that means is, changing each input by a small amount also changes the output by a small amount. And images have lots of pixels. Changing each of them a tiny amount in the right direction adds up to a big change to the output.