GPAI Ledger The public record of EU AI Act training-data summaries

Capture, hash, timestamp: how the evidence is made

Every location where a public summary of training content (Article 53(1)(d), Regulation (EU) 2024/1689) is published gets checked on a daily schedule. This page explains exactly what is stored, how change is detected, and how anyone can verify a capture without trusting this site.

The capture pipeline

  1. Fetch. Each tracked URL is fetched on a daily schedule (conditional GETs spare origins a re-download when nothing changed). JavaScript-only pages are rendered in a real browser and stored as the rendered page, marked as such.
  2. Dedupe. A new version is stored only when content actually changes: byte hash for documents, a whitespace-insensitive text hash for rendered pages, an inner-file hash set for provider bundles.
  3. Prove. Each stored version keeps the exact fetched bytes, the extracted text, HTTP metadata, a SHA-256, an OpenTimestamps proof (anchored in the bitcoin blockchain within hours), and — where the URL permits — a triggered Internet Archive (Wayback) snapshot as an independent witness.
  4. Publish. Every version gets a permanent page; stored bytes are served under their own hash from the content-addressed /blob/ store.

Verify a capture yourself

Files are served under their own hash (e.g. <sha256>.pdf), so the filename is the expected checksum:

sha256sum <sha256>.pdf          # must equal the filename / the version page's SHA-256
ots verify <sha256>.pdf.ots -f <sha256>.pdf   # proves capture time (opentimestamps.org)

OpenTimestamps proofs are attested by public calendar servers within seconds and anchored in the bitcoin blockchain within hours; anchored proofs verify against the blockchain with no trust in this site required. The crawler, verifier, and full corpus (raw bytes, manifests, event log) are public in the project repository, so the whole archive can be re-verified from source.

Permalink stability

Version URLs (/ledger/<provider>/<model>/v/<capture>/) are stable and safe to cite; content-bearing versions are never removed. Model slugs are never renamed. The one exception is narrow: a re-capture whose bytes changed but whose content is identical to a neighbouring version (banner churn, re-rendering) may be pruned as noise — every prune is logged in the append-only event log with the pruned file's hash — and the prune rule itself guarantees the pruned capture's content survives in a neighbouring retained version. Capture ids are minting timestamps and can trail the fetch time by seconds; the manifest's fetched_at is authoritative.

Provider objections

A provider who objects to full-text archiving of a document is switched to a structured-facts treatment: the page keeps the document's hashes, size, text length, and provenance chain — enough to authenticate any copy — without serving the bytes. Write to contact@gpailedger.com.

What the labels mean

provider site / provider page: the provider's own published copy or the page that links it. AIAL archived copy: the write-once snapshot archived by the AI Accountability Lab. watched page: a portal, hub, or listing monitored because documents appear or change there. official document: European Commission material (the template summaries must follow). Code of Practice doc: a document under Art. 53(1)(b)–(c) — related to, but not, the 53(1)(d) summary.

Complementary resources

AIAL's transparency tracker grades the quality of published summaries. Regulation (EU) 2024/1689 on EUR-Lex is the law itself. The European Commission's GPAI pages carry the official template and guidance.