Django&Python, scrape page to fill up database
You can use Celery in your Django project, with Celery you will create tasks to scrape the websites you want and use it inside your Models.
The hardest part should be the scrape it self, insert the data inside your models is easy.
I've got django running with models.
Now, I want that fill todatabase with information from different websites.
There comes scraping, I know. What I'm missing is knowledge if there's any tool to scrape pages with Django that gives compatibility with existing database/models?
Please note that I'd like to run this scraping task every 5 mins for example, so it won't be one-run script.
Could you please point me out which tools should I use? Should I stay with Django or use something external?