Title: I have been scraping housing markets. LinkedIn paired with hunter.io for creating mailing lists of companies. Recently I started MITM and Burp to intercept Apps to get direct api access. I created a docker container using different tor endpoints for different ips. I even used ssh proxying to create my own ip network.<p>To bypass recaptcha I build a container with chrome + puppeteer with anticaptcha preinstalled and pre logged in.<p>Yet I wonder if it is still worth it. Kinda looking for a mentor or inspiration.<p>Been a long time HN reader and I wonder what you think? Why scraping is still worth it in 2018?
Upvote: