Donate to the Internet Archive
I've donated to the Internet Archive and I'm a big fan of Jason Scott, but the Internet Archive <i>is not an archive</i>. Any site on it can go down without warning, thanks to the fact that they apply current robots.txt rules to past archives. Once a domain squatter or regretful admin forbids archivebot (or crawlers in general), archive.org's copy goes down.<p>This has ruined many supposedly permanent links. The infamous "She's a Flight Risk" blog from a decade ago is down.[1] My first website is missing. Even public domain stuff like NASA's report on nuclear propulsion is gone.[2]<p>With just a small rule change (obey robots.txt at the time of crawling), they could eliminate the risk of a page disappearing. Instead, we're stuck with a slower version of the link rot we're used to. It doesn't stop me from supporting them, but it's incredibly frustrating.<p>1. <a href="https://web.archive.org/web/*/http://www.aflightrisk.blogspot.com" rel="nofollow">https://web.archive.org/web/*/http://www.aflightrisk.blogspo...</a><p>2. <a href="http://web.archive.org/web/20121029225832/http://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/19770085619_1977085619.pdf" rel="nofollow">http://web.archive.org/web/20121029225832/http://ntrs.nasa.g...</a>