GPAI Ledger The public record of EU AI Act training-data summaries

Capture, hash, timestamp: how the evidence is made

Every location where a public summary of training content (Article 53(1)(d), Regulation (EU) 2024/1689) is published gets checked on a daily schedule. This page explains exactly what is stored, how change is detected, and how anyone can verify a capture without trusting this site.

The capture pipeline

  1. Fetch. Each tracked URL is fetched on a daily schedule (conditional GETs spare origins a re-download when nothing changed). JavaScript-only pages are rendered in a real browser and stored as the rendered page, marked as such. A 404 seen from the automated runner for a document already archived is not treated as an absence on its own: the target is re-checked after a pause and then cross-checked against an independent witness (a fresh Wayback Machine capture, accepted only if it is genuinely fresh); every check is logged with the vantage point it was made from. An absence is called confirmed only when a second vantage point saw it too — the witness, or the operator checking from another network; a 404 the runner alone keeps seeing on several dates is recorded as persistent, never as confirmed, because one datacenter address cannot tell a removed document from a refused connection.
  2. Dedupe. A new version is stored only when content actually changes: byte hash for documents, a whitespace-insensitive text hash for rendered pages, an inner-file hash set for provider bundles.
  3. Prove. Each stored version keeps the exact fetched bytes, the extracted text, HTTP metadata, a SHA-256, an OpenTimestamps proof (anchored in the bitcoin blockchain; the stored proof is upgraded to its anchor on a later run, a day or two after capture), and — where the URL permits — a triggered Internet Archive (Wayback) snapshot as an independent witness.
  4. Publish. Every version gets a permanent page; stored bytes are served under their own hash from the content-addressed /blob/ store.

Verify a capture yourself

Files are served under their own hash (e.g. <sha256>.pdf), so the filename is the expected checksum:

sha256sum <sha256>.pdf          # must equal the filename / the version page's SHA-256
ots verify <sha256>.pdf.ots -f <sha256>.pdf   # proves the bytes existed no later than the attestation time (opentimestamps.org)

OpenTimestamps proofs are attested by public calendar servers within seconds and anchored in the bitcoin blockchain within hours. This archive re-checks pending proofs on a later run and only then stores the anchored form, so a proof downloaded within a day or two of capture may still read as pending — the attestation it carries is already valid. Anchored proofs verify against the blockchain with no trust in this site required. ots verify checks the attestation against a local Bitcoin Core node (a pruned node is enough); without one, ots info <proof> prints the attesting block height and merkle path, which any block explorer lets you check by hand. The crawler, verifier, and full corpus (raw bytes, manifests, event log) are public in the project repository, so the whole archive can be re-verified from source.

Permalink stability

Version URLs (/ledger/<provider>/<model>/v/<capture>/) are stable and safe to cite; content-bearing versions are never removed. Model slugs are never renamed. Two narrow exceptions exist, and every prune is logged in the append-only event log (tool-made prunes carry the pruned file's hash and reason; the curation-time prunes of August 2026 carry the capture directory, whose hash is in the matching capture event). A re-capture whose bytes changed but whose content is identical to the version before it (banner churn, re-rendering) may be pruned as noise. A capture whose bytes are identical to another capture this archive keeps — which upstream renaming produces, the same file arriving under a new name — may be pruned as a duplicate, and its event names the capture the content survives in. In both cases the prune tool's rule guarantees the removed capture's content survives in a retained version. Capture ids are minting timestamps and can trail the fetch time by seconds; the manifest's fetched_at is authoritative.

Work this project does not republish

Two kinds of capture are held but not served. A provider who objects to full-text archiving is switched to the treatment described next. Separately, a third party's own research — the AI Accountability Lab's scored evaluations of published summaries — is archived so that a grade stays recoverable after they revise it, but their words are not served on this site, in the CC0 dataset or in the change ledger: those pages carry the hashes, size, and timestamp proof, and link to AIAL for the assessment itself. The captures themselves remain in this project's public corpus on GitHub, so that every hash published here can be reproduced from it — withholding applies to what this site and its dataset serve, not to the evidence trail. No objection has been made; the ledger simply does not republish another organisation's work without their permission.

Provider objections

A provider who objects to full-text archiving of a document is switched to a structured-facts treatment: the page keeps the document's hashes, size, text length, and provenance chain — enough to authenticate any copy — without serving the bytes. Write to contact@gpailedger.com.

What the labels mean

provider site / provider page: the provider's own published copy or the page that links it. AIAL archived copy: the write-once snapshot archived by the AI Accountability Lab. watched page: a portal, hub, or listing monitored because documents appear or change there. AIAL evaluation: the AI Accountability Lab's scored assessment of a published summary — their research judgement, not a legal determination, and not the provider's document. official document: European Commission material (the template summaries must follow). Code of Practice doc: a document published under the Art. 56 Code of Practice, whose chapters cover Art. 53(1)(a)–(c) and Art. 55 — related to, but distinct from, the Art. 53(1)(d) summary.

Complementary resources

AIAL's transparency tracker grades the quality of published summaries. Regulation (EU) 2024/1689 on EUR-Lex is the law itself. The European Commission's GPAI pages carry the official template and guidance.