I wrote a web scraper in Scrapy that I call with scrapy crawl scrape_site and a twitter bot in Twython that I call with python twitter.py. I have all the proper files in a directory called ScrapeSite. When I execute these two commands on the command line while in the directory ScrapeSite they work properly.
However, I would like to move the folder to a server and have a job run the two commands every fifteen minutes. I've looked into doing a cron job to do so, but the cronjobs are located in a different parent directory, and I can only call scrapy in a directory with Scrapy files (e.g. ScrapeSite). Can I make a cron job to run a file in the ScrapeSite directory that in turn can call the two commands at the proper level?
How can I programmatically execute command line commands at a different leveled directory at a certain time interval?
Use a cron job. This allows you to run a command line command at a time interval. You can combine commands with && and as a result can change directories with the normal bash command cd.
So in this case you can call cd /directory/folder && scrapy crawl scrape_site && python twitter.py
To run this every fifteen minutes make the cron job run like */15 * * * *
So the full cron job would look like */15 * * * * cd /directory/folder && scrapy crawl scrape_site && python twitter.py
How can I programmatically execute command line commands at a different leveled directory at a certain time interval?