Tinker with a Neural Network in Your Browser
The swiss roll problem also illustrates nicely the idea behind deep learning.Before deep learning people would manually design all these extra features sin(x_1), x_1^2, etc. because they thought it was necessary to fit this swiss roll dataset.
So they would use a shallow network with all these features like this: http://imgur.com/H1cvt8dThen the deep learning guys realized that you don't have to engineer all these extra features, you can just use basic features x_1, x_2 and let the network learn more complicated transformations in subsequent layers.
So they would use a deep network with only x_1, x_2 as inputs:
http://imgur.com/XBRjROPBoth these approaches work here (loss < 0.01). The difference is that for the first one you have to manually choose the extra features sin(x_1), x_1^2, ... for each problem. And the more complicated the problem the harder it is to design good features. People in the computer vision community spent years and years trying to design good features for e.g. object recognition. But finally some people realized that deep networks could learn these features themselves. And that's the main idea in deep learning.