title: How to use scrapy.log module with custom log handler?
body: <p>I have been working on a <a href="http://scrapy.org/" rel="nofollow">Scrapy</a> project and so far everything works quite well. However, I'm not satisfied with Scrapy's logging configuration possibilities. At the moment, I have set <code>LOG_FILE = 'my_spider.log'</code> in the <code>settings.py</code> of my project. When I execute <code>scrapy crawl my_spider</code> on the command line, it creates one big log file for the entire crawling process. This is not feasible for my purposes.</p>
  
  <p>How can I use Python's custom log handlers in combination with the <a href="http://doc.scrapy.org/en/0.14/topics/logging.html#module-scrapy.log" rel="nofollow"><code>scrapy.log</code></a> module? Especially, I want to make use of Python's <a href="http://docs.python.org/library/logging.handlers.html#rotatingfilehandler" rel="nofollow"><code>logging.handlers.RotatingFileHandler</code></a> so that I can split the log data into several small files instead of having to deal with one huge file. The documentation of Scrapy's logging facility is not very extensive, unfortunately. Many thanks in advance!  </p>