Requests-HTML: HTML Parsing for Humans
That's a neat little library. The problem is that the web is rapidly moving away from having HTML as the main information carrier to HTML merely being an envelope to deliver a bunch of JavaScript (and if you're even more unlucky: WebAsm).<p>Scraping the web will become a lot harder in the future.