I currently am developing a website in the Symfony2 framework, and i have written a Command that is run every 5 minutes that needs to read a tonne of RSS news feeds, get new items from it and put them into our database.
Now at the moment the command takes about 45 seconds to run, and during those 45 seconds it also takes about 50% to up to 90% of the CPU, even though i have already optimized it a lot.
So my question is, would it be a good idea to rewrite the same command in something else, for example python? Are the RSS/Atom libraries available for python faster and more optimized than the ones available for PHP?
Thanks in advance,
Jaap
Is solved this by adding a usleep() function at the end of each iteration of a feed. This drastically lowered cpu and memory consumption. The process used to take about 20 minutes, and now only takes around and about 5!
Reading RSS feeds in php or python/something else?