title: Inefficient multiprocessing of numpy-based calculations
body: <p>I'm trying to parallelize some calculations that use <code>numpy</code> with the help of Python's <code>multiprocessing</code> module. Consider this simplified example:</p>
  
  <pre><code>import time
  import numpy
  
  from multiprocessing import Pool
  
  def test_func(i):
  
      a = numpy.random.normal(size=1000000)
      b = numpy.random.normal(size=1000000)
  
      for i in range(2000):
          a = a + b
          b = a - b
          a = a - b
  
      return 1
  
  t1 = time.time()
  test_func(0)
  single_time = time.time() - t1
  print("Single time:", single_time)
  
  n_par = 4
  pool = Pool()
  
  t1 = time.time()
  results_async = [
      pool.apply_async(test_func, [i])
      for i in range(n_par)]
  results = [r.get() for r in results_async]
  multicore_time = time.time() - t1
  
  print("Multicore time:", multicore_time)
  print("Efficiency:", single_time / multicore_time)
  </code></pre>
  
  <p>When I execute it, the <code>multicore_time</code> is roughly equal to <code>single_time * n_par</code>, while I would expect it to be close to <code>single_time</code>. Indeed, if I replace <code>numpy</code> calculations with just <code>time.sleep(10)</code>, this is what I get — perfect efficiency. But for some reason it does not work with <code>numpy</code>. Can this be solved, or is it some internal limitation of <code>numpy</code>?</p>
  
  <p>Some additional info which may be useful:</p>
  
  <ul>
  <li><p>I'm using OSX 10.9.5, Python 3.4.2 and the CPU is Core i7 with (as reported by the system info) 4 cores (although the above program only takes 50% of CPU time in total, so the system info may not be taking into account hyperthreading).</p></li>
  <li><p>when I run this I see <code>n_par</code> processes in <code>top</code> working at 100% CPU</p></li>
  <li><p>if I replace <code>numpy</code> array operations with a loop and per-index operations, the efficiency rises significantly (to about 75% for <code>n_par = 4</code>).</p></li>
  </ul>