title: How to make a scrapy spider run multiple times from a tornado request
body: <p>I have an <code>Scrapy Spider</code> that I need to run when a Tornado <code>get</code> request is called. The first time I called the <code>Tornado</code> Request, the spider runs ok, but when I make another request to the <code>Tornado</code>, the spider does not run and the following error is raised: </p>
  
  <pre><code>Traceback (most recent call last):
      File "/Users/Sebastian/anaconda/lib/python2.7/site-packages/tornado/web.py", line 1413, in _execute
          result = method(*self.path_args, **self.path_kwargs)
      File "server.py", line 38, in get
          process.start()
      File "/Users/Sebastian/anaconda/lib/python2.7/site-packages/scrapy/crawler.py", line 251, in start
          reactor.run(installSignalHandlers=False)  # blocking call
      File "/Users/Sebastian/anaconda/lib/python2.7/site-packages/twisted/internet/base.py", line 1193, in run
          self.startRunning(installSignalHandlers=installSignalHandlers)
      File "/Users/Sebastian/anaconda/lib/python2.7/site-packages/twisted/internet/base.py", line 1173, in startRunning
          ReactorBase.startRunning(self)
      File "/Users/Sebastian/anaconda/lib/python2.7/site-packages/twisted/internet/base.py", line 684, in startRunning
          raise error.ReactorNotRestartable()
  ReactorNotRestartable
  </code></pre>
  
  <p>The tornado method is:</p>
  
  <pre><code>class PageHandler(tornado.web.RequestHandler):
  
      def get(self):
  
          process = CrawlerProcess({
              'USER_AGENT': 'Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)',
              'ITEM_PIPELINES' : {'__main__.ResultsPipeline': 1}
          })
  
          process.crawl(YourSpider)
          process.start()
  
          self.write(json.dumps(results))
  </code></pre>
  
  <p>So the idea is that always that <code>DirectoryHandler</code> method is called, the spider runs and perform the crawling.</p>