title: Fill up a dictionary in parallel with multiprocessing
body: <p>Yesterday i asked a question: <a href="https://stackoverflow.com/questions/38378310/reading-data-in-parallel-with-multiprocess">Reading data in parallel with multiprocess</a></p>
  
  <p>I got very good answers, and i implemented the solution mentioned in the answer i marked as correct. </p>
  
  <pre><code>def read_energies(motif):
      os.chdir("blabla/working_directory")
      complx_ener = pd.DataFrame()
      # complex function to fill that dataframe 
      lig_ener = pd.DataFrame()
      # complex function to fill that dataframe 
      return motif, complx_ener, lig_ener
  
  COMPLEX_ENERGIS = {}
  LIGAND_ENERGIES = {}
  p = multiprocessing.Pool(processes=CPU)
  for x in p.imap_unordered(read_energies, peptide_kd.keys()):
      COMPLEX_ENERGIS[x[0]] = x[1]
      LIGAND_ENERGIES[x[0]] = x[2]
  </code></pre>
  
  <p>However, this solution takes the same amount of time as if i would just iterate over <code>peptide_kd.keys()</code> and fill up the <code>DataFrames</code> one by one. Why is that so? Is there a way to fill up the desired dicts in parallel and actually get a speed increase? i am running it on a 48 core HPC.</p>