title: How to crawl Crunchbase with bot protection (Distil Networks)?
body: <p>Sites like Crunchbase and Glassdoor are all protected by Distil Networks, are there any ways to programmatically ways get data from these sites? I was trying Scrapy+Splash, but somehow they are able to detect this. Are there any other ways to make your requests/javascript validation indistinguishable from a browser?</p>