title: True Concurrency in Python
body: <p>I am relatively new to concurrency in Python and I am working on some code that has to call functions outside of my code. I cannot edit those functions but I need them to run concurrently. I've tried a few different solutions like Multiprocessing, Threading and AsyncIO. AsyncIO comes the closest to what I want if every function I was calling was defined with it, but they're not.</p>
  <p>The functions I'm calling will block. Sometimes for 15-30 minutes. During that time, I need other functions doing other things. The code below illustrates my problem. If you run it you'll see that whether using Threads or Multiprocesses, the tasks always run serially. I need them to run simultaneous to each other. I get that the output blocks until the entire script runs, but the tasks themselves should not.</p>
  <p>What am I missing? With so many choices for concurrency or at least apparent concurrency in Python, I would think this is easier than I'm finding it.</p>
  <pre class="lang-py prettyprint-override"><code>#!/usr/bin/python3
  
  from datetime import datetime
  from multiprocessing import Process
  import sys
  from threading import Thread
  from time import sleep
  
  
  def main():
  
      # Doing it with the multiprocess module
      print(&quot;Using MultiProcess:&quot;)
      useprocs()
  
      print(&quot;\nUsing Threading:&quot;)
      usethreads()
  
  
  def useprocs():
      procs = []
      
      task1 = Process(target=blockingfunc('Task1'))
      task1.start()
      procs.append(task1)
  
      task2 = Process(target=blockingfunc('Tast2'))
      task2.start()
      procs.append(task2) 
  
      task1.join()
      task2.join()
      
      print('All processes completed')
  
  
  def usethreads():
      threads = []
  
      task3 = Process(target=blockingfunc('Task3'))
      task3.start()
      threads.append(task3)
  
      task4 = Process(target=blockingfunc('Task4'))
      task4.start()
      threads.append(task4)   
  
      task3.join()
      task4.join()
  
      print('All threads completed')
  
  
  def blockingfunc(taskname):
      now = datetime.now()
      current_time = now.strftime(&quot;%H:%M:%S&quot;)
      print(current_time, &quot;Starting task: &quot;, taskname)
      sleep(5)
      now = datetime.now()
      current_time = now.strftime(&quot;%H:%M:%S&quot;)
      print(current_time, taskname, &quot;completed&quot;)
  
  
  if __name__ == '__main__':
      try:
          main()
      except:
          sys.exit(1)
  </code></pre>