Format
WARC
Summary
- Name
- WARC
- Version
- 1.0
- Identifiers
-
- MIME: application/warc
- PUID: fmt/1355
- Format type
- Aggregate
- Description
- The WARC (Web ARChive) file format offers a convention for concatenating multiple resource records (data objects), each consisting of a set of simple text headers and an arbitrary data block into one long file. The WARC format is an extension of the ARC file format (ARC) that has traditionally been used to store "web crawls" as sequences of content blocks harvested from the World Wide Web (…) Besides the primary content recorded in ARCs, the extended WARC format accommodates related secondary content, such as assigned metadata, abbreviated duplicate detection events, later-date transformations, and segmentation of large resources”. The WARC format defines no internal compression, however the standard recommends the GZIP format with "deflate" where compression is desirable. WARC format has been written by the members of the IIPC (http://www.netpreserve.org/) grouped within the ISO/TC46/SC4/WG12.
- File extensions
-
warc - Source
- Landesarchiv Nordrhein-Westfalen / Landesarchiv Nordrhein-Westfalen
Relationships
- Has priority over
Internal signatures
WARC 1.0
- Note
- Matching WARC header. Header: WARC/1.0 CRLF. Then either 'WARC-Record-ID', 'WARC-Date', 'WARC-Type', 'Content-Length' all madatory fields that can appear in any order.
Byte sequences
- Min Frag Length
- Absolute from BOF
- Offset
- 0
- Max offset
- 4096
- Byte Sequence
574152432F312E300D0A{0-512}(574152432D5265636F72642D4944|436F6E74656E742D4C656E677468|574152432D54797065|574152432D44617465)- Endianness
- Little-endian
Changelog
-
Added in V97
- Release date
- 1 October 2020
WARC 1.0: Signature developed and samples provided by Landesarchiv Nordrhein-Westfalen.
WARC 1.0: Full entry added. Submitted by Landesarchiv Nordrhein-Westfalen.