Avoiding bot detection: How to scrape the web without getting blocked?
&gt; I need to make a general remark to people who are evaluating (and&#x2F;or) planning to introduce anti-bot software on their websites. Anti-bot software is nonsense. Its snake oil sold to people without technical knowledge for heavy bucks.<p>If this guy got to experience how systemically bad the credential stuffing problem is, he&#x27;d probably take down the whole repository.<p>None of these anti-bot providers give a shit about invading your privacy, tracking your every movements, or whatever other power fantasy that can be imagined. Nobody pays those vendors $10m&#x2F;year to frustrate web crawler enthusiasts, they do it to stop credential stuffing.