Scrapy returning items within spider before first request/parse
Instead of yielding the items, write them into the database directly on the constructor of the pipeline where you add regular items to the database.
I need to return some items from inside a spider before starting to parse requests. Because I need to make sure some parent items exist in database before parsing child items.
I now yield them from the parse method first thing, and this seem to work fine. But I was wondering if there is a better way to do this?