title: Schedule job for SQL
body: <p>I have a list of <code>urls</code> which I need to crawl, so I can index the pages and add them to my <code>searchindex.db</code></p>
  
  <pre><code>urls = ['http://consequenceofsound.net/', 'http://www.tinymixtapes.com/', 'https://www.residentadvisor.net/']
  </code></pre>
  
  <p>this is how I've initialized my crawler class:</p>
  
  <pre><code>class crawler:
      # Initialize the crawler with the name of database
      def __init__(self,dbname):
          self.con = sqlite3.connect(dbname)
  
      def __del__(self):
          self.con.close()
  
      def dbcommit(self):
          self.con.commit()
  </code></pre>
  
  <p>this is the crawling method:</p>
  
  <pre><code>def crawl(self,pages,depth=2):
  
              (...)
              #code here that opens and adds links to database    
              (...)       
  
              self.dbcommit() 
  
              pages=newpages
  </code></pre>
  
  <p>here I instantiate my <code>crawler class</code>:</p>
  
  <pre><code>crawler=crawler('searchindex.db')
  
  pagelist = url[0]
  
  crawler.crawl(pagelist)
  </code></pre>
  
  <p>how do I schedule <code>url crawling</code> and indexing of pages, so that each crawling indexing process resumes after the last one is finished or interrupted by any reason?</p>