What is a good crawling speed rate?
It really depends but you can always check your crawling benchmarks for your hardware by typing scrapy bench on your command line
I'm crawling web pages to create a search engine and have been able to crawl close to 9300 pages in 1 hour using Scrapy. I'd like to know how much more can I improve and what value is considered as a 'good' crawling speed.