title: Spacy, using nlp.pipe on large dataset in python, multiprocessing leads to processes going in sleep state. How to proper use all CPU cores?
body: <p>I'm working on a NLP classification problem over a large database of emails (~1 million). I need to use spacy to parse texts and I'm using the nlp.pipe() method as <code>nlp.pipe(emails,n_process=CPU_CORES, batch_size=20)</code> to loop over the dataset.
  The code works but I'm facing a (maybe not so)weird behavior:
  the processes are being created but they are all in SLEEP state but one, casually some of them go in RUN state for a few seconds and then back to sleep. So I find myself with one single process using one core at 100% but of course the script not using all the CPU cores.
  It's like the processes don't get &quot;fed&quot; input data from pipe.</p>
  <p><em>Does anybody know how to properly use spacy nlp pipe or anyway how to avoid this situation</em>? no way to use nlp.pipe with the GPU?</p>
  <p>Thank you very much!
  Sandro</p>
  <p>EDIT: <em>I still have no solution but i've noticed that if I set the <code>batch_size=divmod(len(emails),CPU_CORES)</code>, the processes all starts running at 100% CPU and after a few seconds they all switch to sleeping state but one. It really looks like some element in spacy pipe gets locked while waiting for something to end.... any idea??</em></p>
  <p>EDIT2: <em>Setting <code>batch_size=divmod(len(emails),CPU_CORES)</code> while processing a large dataset leads inevitably to a spacy memory error:</em></p>
  <blockquote>
  <p>MemoryError: Unable to allocate array with shape (1232821, 288) and data type float32</p>
  </blockquote>
  <p>*thing that is maybe not so weird as my machine has 10GB of RAM and (1232821×288×32)bits / 8 = 1.4GB multiplied by 6 (CPU_CORES) leads to a 8.5GB of RAM needed. Therefore I guess that, having other stuff in memory already, it turns out to be plausible. *</p>