Web ARChive

The Web ARChive (WARC)  archive format specifies a method for combining multiple digital resources into an aggregate archive file together with related information. The WARC format is a revision of the Internet Archive's ARC File Format that has traditionally been used to store "web crawls" as sequences of content blocks harvested from the World Wide Web. The WARC format generalizes the older format to better support the harvesting, access, and exchange needs of archiving organizations. Besides the primary content currently recorded, the revision accommodates related secondary content, such as assigned metadata, abbreviated duplicate detection events, and later-date transformations. The WARC format is inspired by HTTP/1.0 streams, with a similar header and the use of CRLFs as delimiters, making it very conducive to crawler implementations.

WARC is now recognised by most national library systems as the standard to follow for web archival.

Software 

  Heritrix web archiver in Java
 wget (since version 1.14)
 Webrecorder
 StormCrawler
 Apache Nutch
 libarchive

References

External links

 http://archive-access.sourceforge.net/warc/
 http://bibnum.bnf.fr/WARC/
 http://www.digitalpreservation.gov/formats/fdd/fdd000236.shtml
https://netpreserve.org/resources/WARC_Guidelines_v1.pdf
 https://iipc.github.io/warc-specifications/

Category:Archive formats
Category:Web archiving
Category:Web Archives