Machine Learning, Deep Learning, Neural Networks, Optimization, Towards Data Science.
of points. The following graph shows how fast weights get smaller with older values of S compared to threshold = 1 / e, where we mostly ‘forget’ values that are older. The last thing to notice is that the first couple of iterations will provide a pretty bad averages because we don’t have enough