title: Python Script with Gevent Pool, consumes a lot of memory, locks up
body: <p>I have a very simple Python script using gevent.pool to download URLs (see below). The script runs fine for a couple of days and then locks up. I noticed that the memory usage is very high at that time. Am I using gevent incorrectly?</p>
  
  <pre><code>import sys
  
  from gevent import monkey
  monkey.patch_all()
  import urllib2
  
  from gevent.pool import Pool
  
  inputFile = open(sys.argv[1], 'r')
  urls = []
  counter = 0
  for line in inputFile:
      counter += 1
      urls.append(line.strip())
  inputFile.close()
  
  outputDirectory = sys.argv[2]
  
  def fetch(url):
      try:
          body = urllib2.urlopen("http://" + url, None, 5).read()
          if len(body) &gt; 0:
              outputFile = open(outputDirectory + "/" + url, 'w')
              outputFile.write(body)
              outputFile.close()
              print "Success", url
      except:
          pass
  
  pool = Pool(int(sys.argv[3]))
  pool.map(fetch, urls)
  </code></pre>