If i have to choose only one html scraping library for python, which should i choose
Since lots of HTML documents are not well-formed but rather a bunch of tags (sometimes not even properly nested) you probably want to go with BeautifulSoup instead of one of the xml-based parsers.
I need to do lot html parsing / scraping /search engine /crawling.
There are many libraries currently like Scrapy, Beautiful Soup, lxml , lxml2 requests, pyquery.
Now i don't want to try each of these and then decide. basically i want to follow on one and then study in detail and then use that most often.
So which library should i go for which can perform all function mentioned above. Even though there may be diff solutions for diff problems. But i want onelibrary which could do all things even though it takes time to code but should be possible
Is it possible to do indexing in lxml? Is PyQuery same as lxml or its different?