Ask HN: What’s the legality of web scraping?
The current state of the art is hiQ vs LinkedIn:<p><a href="https://www.eff.org/cases/hiq-v-linkedin" rel="nofollow">https://www.eff.org/cases/hiq-v-linkedin</a><p>Basically: if it's publicly visible, you can scrape it.<p>Caveat: the case is still making its way to the Supreme Court.<p>Edit: There's also Sandvig v. Sessions, which establishes that scraping publicly available data isn't a computer crime:<p><a href="https://www.eff.org/deeplinks/2018/04/dc-court-accessing-public-information-not-computer-crime" rel="nofollow">https://www.eff.org/deeplinks/2018/04/dc-court-accessing-pub...</a><p>Edit2: Two extra common sense caveats:<p>- Don't hammer the site you're scraping, which is to say don't make it look like you're doing a denial of service attack.<p>- Don't <i>sell</i> or publish the data wholesale, <i>as is</i> -- that's basically guaranteed to attract copyright infringement lawsuits. Consume it, transform it, use it as training data, etc. instead.