Ask HN: What’s the legality of web scraping?
The current state of the art is hiQ vs LinkedIn:<p><a href="https:&#x2F;&#x2F;www.eff.org&#x2F;cases&#x2F;hiq-v-linkedin" rel="nofollow">https:&#x2F;&#x2F;www.eff.org&#x2F;cases&#x2F;hiq-v-linkedin</a><p>Basically: if it&#x27;s publicly visible, you can scrape it.<p>Caveat: the case is still making its way to the Supreme Court.<p>Edit: There&#x27;s also Sandvig v. Sessions, which establishes that scraping publicly available data isn&#x27;t a computer crime:<p><a href="https:&#x2F;&#x2F;www.eff.org&#x2F;deeplinks&#x2F;2018&#x2F;04&#x2F;dc-court-accessing-public-information-not-computer-crime" rel="nofollow">https:&#x2F;&#x2F;www.eff.org&#x2F;deeplinks&#x2F;2018&#x2F;04&#x2F;dc-court-accessing-pub...</a><p>Edit2: Two extra common sense caveats:<p>- Don&#x27;t hammer the site you&#x27;re scraping, which is to say don&#x27;t make it look like you&#x27;re doing a denial of service attack.<p>- Don&#x27;t <i>sell</i> or publish the data wholesale, <i>as is</i> -- that&#x27;s basically guaranteed to attract copyright infringement lawsuits. Consume it, transform it, use it as training data, etc. instead.