Repository navigation
release: publish-gated append-only provenance ledger for src tarballs - #141
Merged
Merged
Conversation
The rolling releases re-roll a version's src tarball whenever its line's patches move, and a deleted release leaves a recorded release NAME dangling — so the durable provenance of a runtime build input is the tarball's content digest (already recorded downstream as built_from.sources[].sha256; the deterministic-roll rule makes it content-addressed). What was missing is a permanent, verifiable record that the factory published a digest. Every release now carries provenance.yaml: the cumulative, append-only ledger of every (asset, sha256) the factory has ever published, with the release that first carried those bytes. The publish job builds it with tools/provenance --build (chained from the newest existing release's ledger, bootstrapped from every published SHA256SUMS while none carries one) and gates the release on two assertions: - append-only: the previous ledger is an exact prefix of the new one — a re-rolled tarball appends; a published digest is never dropped, reordered, or re-attributed; - completeness: every asset of the release is recorded. Audit path: tools/provenance --verify <sha256> resolves a recorded digest to its asset and first-publishing release (default: the newest release's ledger; --ledger for offline/pinned use); an unattested digest is a named error, exit 1 — loud, never a silent dangle. Tfs::HttpGet gains an optional headers kwarg (the releases API requires a User-Agent; GH_TOKEN/GITHUB_TOKEN authorizes when present).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The dangling case (issue #100), reproduced on main
tebako-runtime-ruby v0.16.11's manifest records its 3.3.12 build input as:
{ "release": "spec22-chain", "sources": [ { "name": "tfs-ruby-3.3.12-src.tar.gz", "sha256": "a54f25657ae219fb5e0cdf1aca5b695320a2963fa8e242b7c106d3d01d2d8f5e" } ] }gh api repos/tamatebako/ruby/releases/tags/spec22-chain→ 404 (deleted chain mirror).surviving release's SHA256SUMS by hand (39 releases) shows v0.2.28 did carry those bytes — the
issue's table predates v0.2.28 by hours. So the digest is verifiable today only by manual
full-history scan, and only while v0.2.28 keeps surviving: nothing stops a re-roll or a
release cleanup from silently orphaning it tomorrow.
The model: provenance by content digest + a cumulative, append-only ledger
The durable reference is the tarball's sha256 — content-addressed by the deterministic-roll
rule and already recorded downstream as
built_from.sources[].sha256. A release name stayswhat it is: fetch metadata. What was missing is a permanent attestation that the factory
published a digest.
Every release now carries
provenance.yaml: the cumulative, append-only ledger of every(asset, sha256)the factory has ever published, each with the release that first carried thosebytes. release-src's publish job builds it (
tools/provenance --build), chained from the newestexisting release's ledger — bootstrapped from every published SHA256SUMS while none carries one
(the one-time recovery path) — and gates the release on:
appends an entry; a published digest is never dropped, reordered, or re-attributed;
Either failure fails the publish with a named error. The ledger chains through the newest
existing release, so factory releases must never be deleted — deleting the newest one breaks
the chain and blocks the next publish loudly (documented in README § "Provenance ledger").
After: the same lookups against the ledger (run live from this branch)
A runtime's recorded
built_from.sources[].sha256now resolves against theprovenance.yamlofits own
built_from.release— or of the newest release; the ledger only ever grows.What changed
tools/lib/tfs/provenance.rb—Tfs::Provenance, the pure ledger model: strict parse/serialize,SHA256SUMS → entries, chaining (first appearance owns
first_release), the append-only andcompleteness assertions, and the audit lookup. Every violation is a named error.
tools/lib/tfs/provenance_feed.rb—Tfs::ProvenanceFeed, the releases-API/download glue(injectable fetcher; drafts excluded, oldest-first).
tools/lib/tfs/http_get.rb— optionalheaders:kwarg (the releases API requires aUser-Agent;
GH_TOKEN/GITHUB_TOKENauthorizes when present). Backward compatible.tools/provenance—--build(publish gate) and--verify <sha256> [--ledger file|-](audit)..github/workflows/release-src.yml— the publish job checks out the repo, builds + checks theledger, and attaches
dist/provenance.yamlto the release.README.md— new "Provenance ledger" section: the model, the two gate assertions, the auditcommand, and the never-delete-releases rule.
Consumer impact (tebako-runtime-ruby)
None required — no field moves.
built_from.sources[].sha256was already the digest; this changemakes it durably verifiable. The fetch path (named assets + SHA256SUMS against the pinned release)
is untouched;
provenance.yamlis an additive asset.Verification
bundle exec rspec— 149 examples, 0 failures (121 → +28: ledger model + feed specs).bundle exec tools/validate_manifests,bundle exec tools/validate_msys_pairs— green.Closes #100