title: How to delay execution within ThreadPool?
body: <p>I've written a script in python using <code>multiprocessing.pool.ThreadPool</code> to handle multiple requests concurrently and do the scraping process robust. The parser is doing it's job perfectly.</p>
  
  <p>As I have noticed in several scripts that there should be a delay within the scraping process when it is created using <strong><em>multiprocessing</em></strong>, I would like to put a delay within my below script as well. </p>
  
  <p><strong><em>However, This is where I'm stuck and can't find out the right position to put that delay.</em></strong></p>
  
  <p>This is my script so far:</p>
  
  <pre><code>import requests
  from urllib.parse import urljoin
  from bs4 import BeautifulSoup
  from multiprocessing.pool import ThreadPool
  
  url = "http://srar.com/roster/index.php?agent_search=a"
  
  def get_links(link):
      completelinks = []
      res = requests.get(link)  
      soup = BeautifulSoup(res.text,'lxml')
      for items in soup.select("table.border tr"):
          if not items.select("td a[href^='index.php?agent']"):continue
          data = [urljoin(link,item.get("href")) for item in items.select("td a[href^='index.php?agent']")]
          completelinks.extend(data)
      return completelinks
  
  def get_info(nlink):
      req = requests.get(nlink)
      sauce = BeautifulSoup(req.text,"lxml")
      for tr in sauce.select("table[style$='1px;'] tr")[1:]:
          table = [td.get_text(strip=True) for td in tr.select("td")]
          print(table)
  
  if __name__ == '__main__':
      ThreadPool(20).map(get_info, get_links(url))
  </code></pre>
  
  <p><strong><em>Once again: all i need to know is the right position within my script to put a delay.</em></strong></p>