Introduction to Artificial Neural Networks – Part 1
I have been messing with ANNs since I learned how to program. For years, understanding of ANNs eluded me. Here is the breakthrough I had some time ago that finally let me understand ANNs:<p>Think about them like circuits. It is easy to see that a circuit full of NAND, AND, OR, etc can carry out computations. An AND gate is a gate that requires all of its inputs to be true and then it goes high. An OR gate requires at least 1 of its inputs to be true and it goes high.<p>Think of a neuron as being somewhere in between that (I understand that in the article he shows how to simulate gates, but doesn't quite describe neurons as being fancy gates (which is where my breakthrough happening)). It requires N of its inputs to be true and it goes high. Then imagine that the "wires" in our neural network don't carry information in the form of bits, but instead in the form of real numbers. Each wire has a "weight" associated with it, such that when the wire is turned on, it outputs that value, rather than a simple binary 1.<p>Now imagine that the neurons take all of these real numbered inputs, and apply some function to them to decide if the neuron wants to turn on or not. It might simply sum them, multiply them, or something more complex, but based on its inputs, it turns on. Its on signal then gets sent to all of the neurons that it points to, etc. The same way you can extract answers from a circuit of logic gates by seeing the output of the gates, you can extract answers from an NN by examining the output of certain neurons.<p>This description is quite simplified and doesn't go into the architecture of ANNs, but if you are really having a lot of trouble grasping how ANNs work, this description should give you some intuition. The hardcore ML people will probably dislike it, but you have to start somewhere. After understanding it like this, I branched out quite a lot and now all of my academic research involves machine learning. But it took that initial breakthrough!