Bypassing website anti-scraping protections
<i>For example, for google.com, you can typically make only around 300 requests per day, and if you reach this limit, you will see a CAPTCHA instead of search results.</i><p>300 is pretty easy to achieve if you're "Googling hard enough" (make 5 slightly different queries, go through the 20 pages of results it's willing to show you, repeat 3 times...), and I've seen it trigger far before that if you are searching for more obscure things. It seems almost hostile to those searching for IC part numbers, specific and very exact phrases, and just "non mainstream" content in general.<p>How sad it is then, that we are told and have internalised the notion that we should use search engines like Google to find things, and yet it prevents us from "trying too hard" to find what we're looking for...