I am trying to build a fast web crawler, and as a result, I need an efficient way to locate all the links on a page. What is the performance comparison between a fast XML/HTML parser like lxml and using regex matching?
You can use pyquery, a library for python that bring you functions from jquery.
Finding links fast: regex vs. lxml