title: Numpy Pure Functions for performance, caching
body: <p>I'm writing some moderately performance critical code in numpy.
  This code will be in the inner most loop, of a computation that's run time is measured in hours.
  A quick calculation suggest that this code will be executed up something like 10^12 times, in some variations of the calculation.</p>
  
  <p>So the function is to calculate sigmoid(X) and another to calculate its derivative (gradient).
  Sigmoid has the property that for <br> <em>y=sigmoid(x), dy/dx= y(1-y)</em> <br>
  In python for numpy this looks like:</p>
  
  <pre><code>sigmoid = vectorize(lambda(x): 1.0/(1.0+exp(-x)))
  grad_sigmoid = vectorize(lambda (x): sigmoid(x)*(1-sigmoid(x)))
  </code></pre>
  
  <p>As can be seen, both functions are pure (without side effects),
  so they are ideal candidates for memoization,
  at least for the short term, I have some worries about caching every single call to sigmoid ever made: Storing 10^12 floats which would take several terabytes of RAM.</p>
  
  <p><strong>Is there a good way to optimise this?<br>
  Will python pick up that these are pure functions and cache them for me, as appropriate?<br>
  Am I worrying over nothing?</strong></p>