title: Clearing memory between memory-intensive procedures in Python
body: <p>I need to sequentially read a large text files, storing a lot of data in memory, and then use them to write a large file. These read/write cycles are done one at a time, and there is no common data, so I don't need any of the memory to be shared between them. </p>
  
  <p>I tried putting these procedures in a single script, hoping that the garbage collector would delete the old, no-longer-needed objects when the RAM got full. However, this was not the case. Even when I explicitly deleted the objects between cycles it would take far longer than running the procedures separately. </p>
  
  <p>Specifically, the process would hang, using all available RAM but almost no CPU. It also hung when <code>gc.collect()</code> was called. So, I decided to split each read/write procedure into separate scripts and call them from a central script using <code>execfile()</code>. This didn't fix anything, sadly; the memory still piled up. </p>
  
  <p>I've used the simple, obvious solution, which is to simply call the subscripts from a shell script rather than using <code>execfile()</code>. However, I would like to know if there is a way to make this work. Any input?</p>