In an RNN to train it, you need to roll it out, and enter in the history of inputs and the history of expected outcomes. 

This doesn't seem like a realistic picture of the brain since this would require, for example, for the brain to store a perfect history of every sense that comes in to it for many time-steps. 

So is there an alternative to RNNs that doesn't require this history? Perhaps storing differences or something? Or storing some accumulator?

Perhaps there is a way to calculate with RNNs that doesn't require keeping hold of this history?