Skip to content

release: publish-gated append-only provenance ledger for src tarballs - #141

Merged
ronaldtse merged 1 commit into
mainfrom
fix/src-provenance
Oct 8, 2026
Merged

ronaldtse merged 1 commit into
mainfrom
fix/src-provenance

Conversation

@ronaldtse

Copy link
Copy Markdown
Contributor

The dangling case (issue #100), reproduced on main

tebako-runtime-ruby v0.16.11's manifest records its 3.3.12 build input as:

{ "release": "spec22-chain",
  "sources": [ { "name": "tfs-ruby-3.3.12-src.tar.gz",
                 "sha256": "a54f25657ae219fb5e0cdf1aca5b695320a2963fa8e242b7c106d3d01d2d8f5e" } ] }
  • The recorded release name dangles outright:
    gh api repos/tamatebako/ruby/releases/tags/spec22-chain → 404 (deleted chain mirror).
  • The recorded digest matches none of the issue's v0.2.21–v0.2.27 window. Scanning every
    surviving release's SHA256SUMS by hand (39 releases) shows v0.2.28 did carry those bytes — the
    issue's table predates v0.2.28 by hours. So the digest is verifiable today only by manual
    full-history scan
    , and only while v0.2.28 keeps surviving: nothing stops a re-roll or a
    release cleanup from silently orphaning it tomorrow.

The model: provenance by content digest + a cumulative, append-only ledger

The durable reference is the tarball's sha256 — content-addressed by the deterministic-roll
rule and already recorded downstream as built_from.sources[].sha256. A release name stays
what it is: fetch metadata. What was missing is a permanent attestation that the factory
published a digest.

Every release now carries provenance.yaml: the cumulative, append-only ledger of every
(asset, sha256) the factory has ever published, each with the release that first carried those
bytes. release-src's publish job builds it (tools/provenance --build), chained from the newest
existing release's ledger — bootstrapped from every published SHA256SUMS while none carries one
(the one-time recovery path) — and gates the release on:

  • append-only — the previous ledger is an exact prefix of the new one: a re-rolled tarball
    appends an entry; a published digest is never dropped, reordered, or re-attributed;
  • completeness — every asset of the release being published is recorded.

Either failure fails the publish with a named error. The ledger chains through the newest
existing release, so factory releases must never be deleted — deleting the newest one breaks
the chain and blocks the next publish loudly (documented in README § "Provenance ledger").

After: the same lookups against the ledger (run live from this branch)

$ tools/provenance --build --tag v0.2.39 --sums <v0.2.38 SHA256SUMS> --out provenance.yaml
provenance: bootstrap over 39 published releases (4553 entries; no previous ledger to prefix-check)
provenance: v0.2.39 adds 0 entries (of 149 assets); ledger now 4553 entries
provenance: append-only + completeness assertions passed; wrote provenance.yaml

$ tools/provenance --verify a54f25657ae219fb5e0cdf1aca5b695320a2963fa8e242b7c106d3d01d2d8f5e --ledger provenance.yaml
a54f25657ae219fb5e0cdf1aca5b695320a2963fa8e242b7c106d3d01d2d8f5e  tfs-ruby-3.3.12-src.tar.gz  (first published: v0.2.28)

$ tools/provenance --verify 38d41d860947cce7e13b509e16acafac8af70a09b7081218dc31a38808372f6f --ledger provenance.yaml
38d41d860947cce7e13b509e16acafac8af70a09b7081218dc31a38808372f6f  tfs-ruby-3.3.12-src.tar.gz  (first published: v0.2.21)
# ^ v0.2.21's bytes, re-rolled away — no current release carries them; the ledger still attests.

$ tools/provenance --verify deadbeef…deadbeef --ledger provenance.yaml
tools/provenance: sha256 deadbeef… is unattested in provenance.yaml: no factory release ever
published these bytes — provenance is unresolvable, do not trust a build input claiming it
(exit 1)

A runtime's recorded built_from.sources[].sha256 now resolves against the provenance.yaml of
its own built_from.release — or of the newest release; the ledger only ever grows.

What changed

  • tools/lib/tfs/provenance.rb — Tfs::Provenance, the pure ledger model: strict parse/serialize,
    SHA256SUMS → entries, chaining (first appearance owns first_release), the append-only and
    completeness assertions, and the audit lookup. Every violation is a named error.
  • tools/lib/tfs/provenance_feed.rb — Tfs::ProvenanceFeed, the releases-API/download glue
    (injectable fetcher; drafts excluded, oldest-first).
  • tools/lib/tfs/http_get.rb — optional headers: kwarg (the releases API requires a
    User-Agent; GH_TOKEN/GITHUB_TOKEN authorizes when present). Backward compatible.
  • tools/provenance — --build (publish gate) and --verify <sha256> [--ledger file|-] (audit).
  • .github/workflows/release-src.yml — the publish job checks out the repo, builds + checks the
    ledger, and attaches dist/provenance.yaml to the release.
  • README.md — new "Provenance ledger" section: the model, the two gate assertions, the audit
    command, and the never-delete-releases rule.

Consumer impact (tebako-runtime-ruby)

None required — no field moves. built_from.sources[].sha256 was already the digest; this change
makes it durably verifiable. The fetch path (named assets + SHA256SUMS against the pinned release)
is untouched; provenance.yaml is an additive asset.

Verification

  • bundle exec rspec — 149 examples, 0 failures (121 → +28: ledger model + feed specs).
  • bundle exec tools/validate_manifests, bundle exec tools/validate_msys_pairs — green.
  • Live bootstrap against the production releases API (the output above).
  • First real release after merge bootstraps and attaches the ledger; every later release chains.

Closes #100

The rolling releases re-roll a version's src tarball whenever its line's
patches move, and a deleted release leaves a recorded release NAME
dangling — so the durable provenance of a runtime build input is the
tarball's content digest (already recorded downstream as
built_from.sources[].sha256; the deterministic-roll rule makes it
content-addressed). What was missing is a permanent, verifiable record
that the factory published a digest.

Every release now carries provenance.yaml: the cumulative, append-only
ledger of every (asset, sha256) the factory has ever published, with the
release that first carried those bytes. The publish job builds it with
tools/provenance --build (chained from the newest existing release's
ledger, bootstrapped from every published SHA256SUMS while none carries
one) and gates the release on two assertions:

- append-only: the previous ledger is an exact prefix of the new one —
  a re-rolled tarball appends; a published digest is never dropped,
  reordered, or re-attributed;
- completeness: every asset of the release is recorded.

Audit path: tools/provenance --verify <sha256> resolves a recorded
digest to its asset and first-publishing release (default: the newest
release's ledger; --ledger for offline/pinned use); an unattested digest
is a named error, exit 1 — loud, never a silent dangle.

Tfs::HttpGet gains an optional headers kwarg (the releases API requires
a User-Agent; GH_TOKEN/GITHUB_TOKEN authorizes when present).
@ronaldtse
ronaldtse merged commit c877da6 into main Oct 8, 2026
60 checks passed
@ronaldtse
ronaldtse deleted the fix/src-provenance branch October 8, 2026 15:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

src tarballs regenerate per rolling release; runtime provenance (built_from.release=spec22-chain) dangles

2 participants