Scrapy - adding an unique ID to scraped data
If you are running on Scrapinghub Cloud, you can call the API to get the latest job ID when running, and the ID will be unique for the job.
I am scraping a list of products every day using Scrapy but the products do not have unique ID numbers. So I have tried to manually assign 1,2,3,4 in the CSV file but this is not working since the products are changing from 1 to 5 for example if the site changes. Do you have any other solution that you can think so when I upload to my database the products will not duplicate?
Also, is there a way to ask Scrapy to automatically add 1,2,3,4 automatically?
Cheers.