Ask HN: What’s the legality of web scraping?
The current state of the art is hiQ vs LinkedIn:https://www.eff.org/cases/hiq-v-linkedinBasically: if it's publicly visible, you can scrape it.Caveat: the case is still making its way to the Supreme Court.Edit: There's also Sandvig v. Sessions, which establishes that scraping publicly available data isn't a computer crime:https://www.eff.org/deeplinks/2018/04/dc-court-accessing-pub...Edit2: Two extra common sense caveats:- Don't hammer the site you're scraping, which is to say don't make it look like you're doing a denial of service attack.- Don't sell or publish the data wholesale, as is -- that's basically guaranteed to attract copyright infringement lawsuits. Consume it, transform it, use it as training data, etc. instead.