Is there a way to have more than 2 spiders crawling different websites in one project using Scrapy?
In theory both is possible, you can have multiple spiders in one project or set up multiple projects. However with 10 websites chances are quite high that you need distinct pipelines, middleware-settings and so on. For that reason I would generally suggest using multiple projects. Also your items.py might grow very large with multiple spiders and so you might loose track.
I have around 10 websites I need to scrape some data off from. I'm using Scrapy, and I need to know if I have to make 10 separate projects (one for each website) or 10 spiders in a single project?